Team Foundation Services

We have one solution that manages our DW ETL. Every table is it's own
package. So we have about 18 packages.

What is hard to manage is we have 5 developers working on various packages
and when they check in, Team Foundation Services rarely merges the code
happily. What ends up happening is the clashing developers, whoever did the
least work, allows the other to be master and they re-do their code.

Does SSIS not work well with TFS and multiple developers? Don't we have to
allow multi-checkouts because when you work on any given package, TFS not
only checks out the package but also the Solution. If we didn't allow multi
check-outs - only one developer could work on any one solution at one time,
right?

We are now in release stages and have hot fixes - what we really need to do
is branch - but I can't imagine the trauma of merging 4 to 6 weeks worth of
changes across many packages when it's such a struggle with just a couple
days worth of changes.

What are others doing?

You'll be glad to know I had to wrestle with this problem at my last shop.
However, there is no purely technical solution for this problem afaik. One
of the disadvantages of SSIS is that its XML-based and whilst this means we
get to use GUI's etc. rather than delving around with code, it also means
branching and merging become more onerous. This goes for all of the popular
source control products I've used, not just TFS.

Firstly I'd advise branching your SSIS project per release, with a Trunk
that represents the production packages. That way different developers can
work on different releases without getting check-out conflicts (although you
can set TFS to use 'shared' check-outs I'd advise against it!). I'll come
onto managing cascading the changes through branches in a moment. You
shouldn't need to check out the solution or project file every time to
simply change a package that already exists in the solution or project, so
make sure developers aren't doing it by default.

Hot-fixes can be applied directly to the trunk or to a very short-lived
branch that gets merged straight back into the trunk once the fix is live.

To ensure that changes to the trunk, or branches are cascaded through to
subsequent branches the most effective method we found had two elements.
Firstly we nominated a developer to be responsible for ensuring all updates
to the trunk or previous branches are applied to 'his' (or her!) branch.
This may or may not be the developer doing the work themselves - hopefully
your devs are happy to share the pain! However the developer responsible has
to 'own' the changes. Secondly we used a feature in TFS which allows an
email alert to be sent when a check-in is made on a branch or trunk, along
what was checked-in, who made the check-in etc. Every morning each dev
responsible for changes to their branch checks his mail to see what
check-ins have taken place. If a check-in has happened on a previous branch
he has to go across to the developer who made the check-in and find out what
changed. This worked pretty well with a team of 4-5 developers with up to 3
branches and a trunk. Using mail-box rules can also filter out irrelevant
mails. You can also apply this method to other parts of the BI stack that
are xml-based such as Analysis Services and Reporting Services.

The important thing here is to manage the changes on a daily basis -
otherwise as you say the amount of work involved could be overwhelming. In
your position I'd try and apply the hotfixes before branching and then
establish a baseline (ie trunk) to branch from. Then use the techniques
above to manage the changes every day.

I'd advise against attempting to merge packages using an XML editor (such as
XMLSpy) or the native TFS comparison tool as you can wind-up with a
corrupted package all to easily. However BIDS helper
(http://www.codeplex.com/bidshelper) does include a Smart Diff tool
(http://www.codeplex.com/bidshelper/W...ringTitle=Home)
which takes some of the pain out of comparing packages, by tidying it up a
bit and removing less relevant formatting from the comparison. Use this to
work out changes when the developer who made the change can't remember what
he did, or went on holiday etc. Still make the changes in BIDS itself. This
is also good for comparing branches to the trunk once a release is live,
prior to merging into the trunk and dropping the branch (ie there shouldn't
be any differences).

Other techniques can help mitigate contention while developing packages (ie
multiple projects, multiple packages per table) but to my mind these contain
flaws (unnecessary staging between packages for example).

If this advice is any good I might add it to my blog, so I hth.

--
Phil

http://www.clarity-integration.com
http://www.phil-austin.blogspot.com

"Joe H" wrote in message
news:#6p$GluAJHA.3756-at-TK2MSFTNGP03.phx.gbl...
> We have one solution that manages our DW ETL. Every table is it's own
> package. So we have about 18 packages.
>
> What is hard to manage is we have 5 developers working on various packages
> and when they check in, Team Foundation Services rarely merges the code
> happily. What ends up happening is the clashing developers, whoever did
> the least work, allows the other to be master and they re-do their code.
>
> Does SSIS not work well with TFS and multiple developers? Don't we have
> to allow multi-checkouts because when you work on any given package, TFS
> not only checks out the package but also the Solution. If we didn't allow
> multi check-outs - only one developer could work on any one solution at
> one time, right?
>
> We are now in release stages and have hot fixes - what we really need to
> do is branch - but I can't imagine the trauma of merging 4 to 6 weeks
> worth of changes across many packages when it's such a struggle with just
> a couple days worth of changes.
>
> What are others doing?
>

On 20-Aug-2008, "Joe H" wrote:

> We have one solution that manages our DW ETL. Every table is it's own
> package. So we have about 18 packages.
>
> What is hard to manage is we have 5 developers working on various packages
>
> and when they check in, Team Foundation Services rarely merges the code
> happily. What ends up happening is the clashing developers, whoever did
> the
> least work, allows the other to be master and they re-do their code.
>
> Does SSIS not work well with TFS and multiple developers? Don't we have
> to
> allow multi-checkouts because when you work on any given package, TFS not
> only checks out the package but also the Solution. If we didn't allow
> multi
> check-outs - only one developer could work on any one solution at one
> time,
> right?
>
> We are now in release stages and have hot fixes - what we really need to
> do
> is branch - but I can't imagine the trauma of merging 4 to 6 weeks worth
> of
> changes across many packages when it's such a struggle with just a couple
> days worth of changes.
>
> What are others doing?

Just FYI, the project file should not be checked out just because you are
editing an SSIS package. Only if you are adding a new SSIS package should
this happen.

Visual Studios auto checks out the project every time is there a way to not
have it auto do this - even just looking at a package causes this behavior.
"mouser" wrote in message
news:6GaDk.5$Wc3.0-at-fe091.usenetserver.com...
>
> On 20-Aug-2008, "Joe H" wrote:
>
>> We have one solution that manages our DW ETL. Every table is it's own
>> package. So we have about 18 packages.
>>
>> What is hard to manage is we have 5 developers working on various
>> packages
>>
>> and when they check in, Team Foundation Services rarely merges the code
>> happily. What ends up happening is the clashing developers, whoever did
>> the
>> least work, allows the other to be master and they re-do their code.
>>
>> Does SSIS not work well with TFS and multiple developers? Don't we have
>> to
>> allow multi-checkouts because when you work on any given package, TFS not
>> only checks out the package but also the Solution. If we didn't allow
>> multi
>> check-outs - only one developer could work on any one solution at one
>> time,
>> right?
>>
>> We are now in release stages and have hot fixes - what we really need to
>> do
>> is branch - but I can't imagine the trauma of merging 4 to 6 weeks worth
>> of
>> changes across many packages when it's such a struggle with just a couple
>> days worth of changes.
>>
>> What are others doing?
>
> Just FYI, the project file should not be checked out just because you are
> editing an SSIS package. Only if you are adding a new SSIS package should
> this happen.