Recently I was working with Dependency Replicator and I ran into an issue.  

 

Background

The file that is the "dependency" does not exist in the destination yet. Therefore the replicator fails. If it does not find the destination file in the dependent project location it will fail. In English: In $/TeamProject/Main/SharedBinaries/ there was no "utility.dll". However, this is EXACTLY where I wanted replicator to PUT the file. Since the file did not exist at the destination, I would receive this error in the event log:

-----------------------------------
Error in Event Viewer:

Dependency Repliccator error:
Event Type: Error
Event Source: DependencyReplicator
Event Category: None
Event ID: 0
Date:  10/23/2008
Time:  11:24:30 AM
User:  N/A
Computer: TFSSERVER
Description:
An error occured processing the dependency '$/TeamProject/Path/To/Dependency/dependency.dll' from build 'TeamProject_CI_20081023.5' in project 'TeamProject'. Processing subsequent rules will continue:
System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\DependencyReplicator\TeamProject\Path\To\Dependency\Dependency.dll'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.File.InternalCopy(String sourceFileName, String destFileName, Boolean overwrite)
   at DependencyReplicator.BuildStoreWatcher.ExecuteDependencyReplicationRule(Rule rule, Workspace workspace)
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

---------------------

 

Steps to recreate:

1. Create two team projects
   1.   -  One being the one which will propogate the dependency (ex: shared utility dll or something). This would output a file called "utility.dll" which would then
        be propogated to other locations.
   2. Other being the project which utilizes the dependency "MytestProject". DO NOT put the dependency (utility.dll) in this team project anywhere.
2. Set Depdenency Replicator xml to have the utility.dll file placed somewhere in "MyTestProject".
3. Kick off a build for the utililty project. This should kick off the dependency repliccator.
4. Error (above) occurs.

 

Work Around

Place a copy of the "utility.dll" into the destination where the dependency should be. From that point forward you will not have a problem.

 

The Real Fix

The real fix is to fix the source code. Since I'm not a developer on the project I whipped up this code (which please note is only POC at this time).

 

string fullyQualifiedSource = Path.Combine(rule.Event.Data.DropLocation, rule.Source);
string fullyQualifiedDestination = workspace.GetLocalItemForServerItem(rule.Destination);
if(status.NumOperations != 0) 
{
workspace.PendEdit(rule.Destination); 
File.Copy(fullyQualifiedSource, fullyQualifiedDestination, true);
}
else
{
File.Copy(fullyQualifiedSource, fullyQualifiedDestination, true);
workspace.PendAdd(rule.Destination); 
}

I'm going to build this hopefully later this week and submit it to the project. :)

This is kind of a "known thing" to those of us who automate team builds tasks in TFS with build scripts but there are still a lot of people who don't know this... so here we go...

The 2008 version of TFS implemented Continuous Integration. In TFS, a build is triggered upon every check-in (if you have that option enabled in TFS). This can be setup in the build definition as shown below:

 

 

When anyone  checks a file into the workspace that is defined within the build (see the "Workspace" area in the image above), TFS will kick off a build.

Sometimes you'll need to automate your build through some custom MSBuild tasks and during that time you may need to check a file back into TFS after a build. Why? Perhaps you're generating some code on the fly, based upon the output of the  build. Perhaps you're generating Xml documentation and you need to drop that back into the source repository for whatever reason. When you check code back into TFS, TFS will kick off a build, quickly leading to an endless loop.

 

How To Get Around This

Place the following into the comment field: ***NO_CI*** You can also use the $(NoCICheckinComment) Property in a build file. This property is set at runtime when MSBuild is started from the TFS Build agent.

The "***NO_CI***" comment will instruct team build to ignore this changeset as a trigger for continuous integration.

With continuous integration being integrated into Team Foundation Server 2008 we can sometimes run into unexpected results. Such as extended build times.  By default the build file is set up to inform the TFS Build Agent that it should perform a "Full Build".

What is a "Full Build"?

By default, when TFS performs a build the following events happen:

1. Clean - When Team Foundation Build performs a full build, it cleans the files in the intermediate build folder and sources folder in Clean target.
2. Workspace Initialization - As a part of a full build, Team Foundation Build deletes the old, and recreates a new workspace to clean and synchronize the source files.
3. Get - In the Get task, Team Foundation Build by default retrieves new source files and overwrites the existing files in the build workspace.

 

A Possible Issue

A full build can have consequences that can be considered "non-optimal" (this is very subjective in many circles), but in my opinion coming from a Cruise Control .NET background, I DO NOT want full builds.

If my source code repository is very large, cleaning the files, re-creating the workspace and then getting all the files again can be a real pain in the "time" category. I've had builds on my local dev machine (which is in a VPC) take 15-20 seconds to build using Visual Studio. I've even run the same build from the command line using MSBuild using the .proj file and its taken less than 15 seconds for he same build (~15 Visual Studio Projects in the solution).

However, this same build using TFS would take nearly 20+ minutes!

Wowzers... The reason the build took so long in TFS is because it was utilizing the FULL Build.

 

 

 

 

Core Concept

One of the core concepts is "Keep The Build Fast". We're looking for frequent feedback. 20 minutes (IMO) is not quick feedback (if the build takes 15-20 seconds on my machine). We should be able to get this down to under 10 minutes at the worst, but my goal is to get it well under 5 minutes (closer to 2 minutes).

 

Enabling TFS 2008 Incremental Builds

In Team Foundation Server 2008, we can change this behavior with a simple change in the TfsBuild.proj file under the TeamBuildTypes directory in TFS. Here's how:

1. Open the TfsBuild.proj file for the build in question
2. Add the following XML:

This will enable Team Build to treat this build as an incremental build.

What does this exactly do behind the scenes?

IncrementalBuild is a convenience property that sets SkipClean to true, SkipInitializeWorkspace to true, and ForceGet to false.

Since clean is skipped, the files will not be deleted. Since SkipInitailizeWorkspace is skipped, the workspace will not be deleted and recreated and since ForceGet is disabled, the files will not be overwritten with the files from Source Control. What this will do is perform a regular "Get" on the existing workspace which will only get the changed files.

 

Result

A huge performance improvement. The previous ~20 minute build dropped to about 4 minutes. There are some other improvements I could do to improve the performance,but at this time its 5 times as fast. Excellent. :) Why did it improve this much? Simple: Less disk I/O. Disk I/O = slow.

 

Disclaimer: This may not be desired. For example, build that is specified for "Production" you might want a "Full Build" in order to simulate a production release. You may have tasks in your build file that need a clean build. But for Continuous integration on the project, this setting is highly desirable.

I forgot where I learned this (perhaps the TFS Admin guide) but ... this is a good little tip...

 

When you add users to groups in AD and those groups are present in the Team Foundation Server Security groups it can take up to 60 minutes for the policy to be pushed down into TFS.

To jumpstart this, restart the "Visual Studio Team Foundation Server Task Scheduler".

 

After the service restarts it kicks off the services that are responsible for refreshing the security policies. Now, if you add a user and they need immediate access, you can do it. Beware of other users though, this could impact your system if someone is checking in files, or working with TFS. So take caution in doing this.

My plan today was to upgrade Team Foundation Server 2008 to Service Pack 1 (SP1) today. My client runs a dual server implementation and we run Team Explorer on the App Tier of this set up. Therefore we needed to upgrade Team Explorer and TFS itself to SP1 for 2008.

 

Round 1

The TFS SP1 Documentation states the following -

"...you should first install SP1 for Visual Studio 2008 and then install SP1 for Team Foundation Server on any server that is running both Team Foundation Server and Team Explorer. If you try to update Team Foundation Server first, the update will fail. If you update Team Foundation Server but not Team Explorer, you will encounter errors if you try to use that installation of Team Explorer to perform certain administrative tasks on Team Foundation Server. "

Makes sense. Install SP1 for Visual Studio 2008 and then move to SP1 for TFS. So we did...

Upon installing SP1 for Visual Studio we had to reboot (for whatever reason) ... upon reboot services failed to start and we were notified. Upon further investigation in the event viewer we found the following entries:

"The configuration data for this product is corrupt."

and

"TF30059: Fatal error while initializing web service ..."

and

"TF53010: The following error has occurred in a Team Foundation component or extension: ..."

Plus, inside of the event viewer I was seeing that TFS was "Crashing" and a crash report was being generated for Microsoft. I did some searching and found this blog entry (thanks Tom) and noticed that he was having a very similar problem. The blog recommended that we install SP1 for TFS as this was the hot-fix as listed here by Microsoft. Apparently there is a problem with SP1 Explorer components and NON-SP1 TFS components.

No problem, right? Lets install SP1 for Team Foundation Server 2008. Nah, nope, didnt work: FATAL ERROR. Again, baffled, back to the drawing board.

 

Round 2

Upon further looking into the service errors it was brought to my attention that the IIS Admin service was NOT starting. Running "iisreset" from the command line didn't do anything either (other than giving me a FAIL finger). It would fail stating that one of the dependencies wasn't starting up. I looked up the dependencies and noticed this:

I also noticed that the service Dependency Replicator was failing as well. Ok, so this makes sense... if Depdendency replicator failes, then everything else fails in the dependency tree. I had installed this service to help with some continuous integration stuff I'm doing for the client.

 

The Solution...

I removed this service (completely uninstalled it). Then tried to start IIS. It worked. Cool.

We then tried to start the TFS Services. Nope. Failure still. But now, the symptoms were EXACTLY the same as noted this blog entry.

We then kicked off the TFS 2008 SP1 installation and then tried to start the TFS Services again. SUCCESS!

I then re-installed Dependency Resolver and everything started working as normal. No errors.

 

Root Cause

In the TFS installation guide the following is noted:

"... If you update Team Foundation Server but not Team Explorer, you will encounter errors if you try to use that installation of Team Explorer to perform certain administrative tasks on Team Foundation Server... "

Essentially the guide is stating "if you update one, but not the other, then it won't work". This is exactly what happened. I haven't tapped into the TFS Dependency Replicator Code base that much but I assume that it uses some components that are shared between Explorer and TFS. When this mis-match occurred it broke.

 

RE-Cap - Events & HOW TO FIX

1. Installed VS 2008 SP1 to update Team Explorer
2. Tried to install TFS 2008 SP1 to update TFS
3. Fatal Error Occurred.
4. Traced back the dependencies to Dependency Replicator was blocking the web publishing (seen above)
5. Stop and Uninstall Dependency Replicator
6. Install TFS 2008 SP1
7. Restart
8. Re-Install Dependency Replicator
9. Done

Yup, that's right. Team Foundation Sever told me TOO BAD, SO SAD, GO AWAY, TRY AGAIN ANOTHER DAY. What a great way to start a Monday. Oh, any by the way, we have an emergency release that might have to get pushed out. We're waiting on upper management to make the final decision... fun-fun it is.

Photo By: Auntie P

This weekend I rebuilt my machine. I have a Dell Inspiron 6400 Laptop and it came loaded with Vista Business (and TONS of junk-ware I don't need/nor care about). The one thing I love is that to restore back to factory settings only took about 10 minutes. Yes, 10 minutes - that's craziness. Most rebuilds take hours (unless you're using Ghost or something like that). There is a little partition on the drive that has the factory restore files on it. Just boot into the boot menu during start up (F8) and then choose restore to factory default and you'll have a factory machine within 10 minutes, total awesome-ness.

After I spend the next 6-8 hours installing VS2005, VS2008 (yes, I need both, some clients will only have VS2005 and the solution files in VS2005 and VS2008 are different, by a line), Office 2007, and my list of tools that are close to Hanselmans tools list I finally was ready to get back to work on Monday. I came in today and tried to connect to TFS (2005). Nope. Not going to happen. Here's the error:

 

TF31002: Unable to connect to this Team Foundation Server (...) Please contact your TFS Administrator.

 

Me: *Expletive*

Great, I am the TFS administrator. I have FULL access rights to TFS, I AM THE ADMINISTRATOR!! I have no idea why this would happen. I was able to connect last Friday, on Saturday before the re-install and team members are working against TFS as I speak. Then I rebooted, tried a few other things. Checked Windows firewall and a few other places. Nope. Still didn't work.

Me: *Expletive*

 

The Fix

I had a hunch this was caused by some type of interception of a call to TFS. I then decided to uninstall Macfee Anti-Virus because, well, because I can't stand anti-virus software that bloats my system. After the always-required-reboot, I started up VS2005 and tried to connect to TFS.

It worked.

I wanted to know if anyone else had this problem so I did a little poking around and I found this thread on the MSDN forums. It looks like this is the EXACT problem that I had.

Oh what a joy AV software is. Actually, I can't stand it - not one bit.

Oh what a P.I.T.A. this error has been. I received this error today...

Microsoft.TeamFoundation.VersionControl.Client.ItemExistsException: The item $/path/to/myFile.cs already exists. ---> System.Web.Services.Protocols.SoapException: The item $/path/to/myFile.cs already exists. --- End of inner exception stack trace --- at Microsoft.TeamFoundation.VersionControl.Client.Repository.ProcessHttpResponse(HttpWebResponse response, Stream responseStream, WebException webException, XmlReader& xmlResponseReader) at (...)

blah blah blah...

We have two build servers set up at the client where I'm at and I'm in charge of keeping them up and running smoothly.

Build Server 1: Local, same office as where our TFS Server is

Build Server 2: Remote, connects via TFS Proxy Server at the remote location which connects to our TFS server (next to Build Server 1)

 

PROBLEM 

---

The problem is that I recently moved a project using the Move command in TFS.

The project was moved from a location which was incorrect for our Continuous Integration environment to a location that that was optimal.

Between the time that this move occurred and when the project was altered again, a file was dropped from TFS somehow yet it remained in the project. Not a big deal right? I asked a coworker to add it through Team Explorer. Bam, no problem, done. The file was back, it built correctly on our development machines and we went onto the next task.

Then CC.NET Barfed. It was the error that is in red above.

Robert Horvick had some great insight into this error on this post. Here's what he says:

"At a high level what is happening here is that there are two distinct items which happen to have the same path name (at different points in time, obviously).  When you are labeling TFS sees what you want to put into the label (the new item) and what is already in the old label (the old item) and sees the name conflict.  Since /child:replace means to replace the label on the same item it can't drop the label off the old item which means it can't add it to the new item so it issues this error message."

THANK YOU Robert.

 

SOLUTION

---

As Robert also stated in that post we have to "unlabel" the file so CC NET can re-label it correctly. We're getting rid of the old label on this file. We do this with the "tf unlabel" command.

tf unlabel 1234 "$/Path/to/myFile.cs"

(where 1234 is the label)

I did this, and then forced a build through CC.NET and it worked! Woo hoo!

UNTIL... the next build, it broke, and I get the same exact error message above.

 

After some research I came to find out that this solution that was getting labeled was a solution that was getting labeled multiple times a day on two separate build servers. We have a common source solution that is shared among many applications (such as security objects, loggers, etc). So while I cleared the label on the one build server, there was still a label that might have been newer on the new build server.

In order to get around this, I had to clear all the labels for that file. Normally this is not something I'd ever want to do, but fortunately this file only had about 4 labels applied to it as it was fairly new and we were not doing anything with those said labels (just cc.net labels). After removing the labels, everything started building just fine.

_PublishedWebsites does not exist?! WHAT?! Huh?

One of our TFS builds was succeeding and a fellow developer noticed that the Web App was not getting copied into the _PublishedWebsites folder of our drop location.

I took a look at our TFSBuild.Proj file and I didn't see anything that stood out. I took a look at a similar project who's output was being copied into the _PublishedWebsites directory and the .proj file matched that of the one that did not work.

Solution

The problem was not in the .proj file, but in the actual .csproj file for the Web Applicaiton Project.

.csproj files ARE MSBuild files and inside of that I saw this:

<Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
<!--<Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v8.0WebApplications\Microsoft.WebApplication.targets" />-->

The second Build target file was not getting loaded.

This build target file contains a target by the name of "_CopyWebApplication" which takes the output of the web application and copies it to the _PublishedWebsites directory.

I uncommented this line, checked it back into TFS and re-ran my build and my _PublishedWebsites started showing up. :)

My file now looks like this:

<Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
<Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v8.0WebApplications\Microsoft.WebApplication.targets" />