Continuous Integration is a software development practice where members of a team integrate their work frequently; usually each person integrates at least daily - leading to multiple integrations per day. - Martin Fowler 

Martin Fowler’s article “Continuous Integration” listed several key practices that make up “Effective CI”. Here is a quick overview of how Team Foundation Server combined with Automaton can help implement some of these practices.

Key Practices identified by Fowler:

1. Maintain a Single Source Repository

2. Automate the Build

3. Everyone Commits Every Day

4.  Automate Deployment

Side Note:

Best Practices for setting up projects in Team Foundation Server

Taken from TFS Best Practices Guide

- Decide on team project structure
- Understand the culture and requirements for each of the teams
- Map requirements to our logical and physical guidance and create first draft of the logical
- Make necessary modifications to the structure (baseless merge where appropriate)
- Finalize logical and physical diagrams
- Follow release scenarios on these diagrams
- Create Team Project and physical version control structure (import any existing code)
- Create best practices and provide training
- Learn from experiences, customize the branch structure, and make it yours

1: Maintain a Single Source Repository

Team Foundation Server’s source control service provide branching and merging capabilities that help to ensure a centralized and controlled repository for all items needed to build the application successfully. Figure 2 details the specific folder structure and branching strategy used at a former company.

Figure 2 Source Control Structure

2: Automate the Build

A build process helps ensures that the code will compile independently from the developer’s workspace and standardizes configuration across the promotion process. Oftentimes developers will forget to check in critical files and then deploy from their computer. After the initial release everything may run fine but when the next developer checks out the source to make modifications they soon discover that the application no longer builds.

• Determine promotion process needed for project (dev, staging, release)
• Create a Build Type on team foundation server for each promotion type
• If .Net is used configure build types for each solution
• If a website is used configure web deployment project for each build type (this is where dev config files and production config files are specified)
• If the application is an access or .asp application then configure .bat files or other script files. These will execute during the build process (this may run testing against code or verify that certain dll’s are registered)

3: Everyone Commits Every Day

Checking in code is done as soon as a new task is completed. If a task is carried over to the next day the item is “shelved”.

Figure 3 Check-In - Build - Deploy

4: Automate Deployment

note: since the time I had documented this we have upgraded to TFS 2008 which has made CI simple and eliminated the need for Automaton. 

At a previous company one of our goals was to have an integration process that allowed us to provide updates to an application as real-time as possible. The deployment process was setup to build and deploy to the appropriate server as soon as a developer checks-in code. This allowed us to push out new features and bug fixes without having to worry about the manual process of copy files and testing to make sure the appropriate references have been set.

• for web projects this involves configuring the web deployment project for the solution to deploy the compiled source to the correct server for each build type
• for non .net solutions like access or asp this may involve creating bat files that are executed by the build server to copy the necessary files to each server

The key to making the deployment “Real-Time” is to either write custom code that hooks into Team Foundations Servers “Check-in” event or use a third party application that does that for you. In our case we use Automaton.

Automaton is configured to run the specified build configuration when code is checked in. You can also monitor the build status or view the history of the builds using their web interface.

Web deployment projects can be a useful addition to your solution as well. Staging and release configurations can be controlled during the automated deployment process (i.e. web.config file that has dev, staging, and production settings).

Figure 4 third party TFS add on

Credits:

Martin Fowler http://www.martinfowler.com/articles/continuousIntegration.html

Trying to find the MVC framework’s place in the development world view of Web 2.0 and EXTJS (or any other JavaScript library) I am reminded of Microsoft's 2000 PDC unveiling of ASP.NET and the announcement of the fairy dust of integration, web services (you just need a sprinkle). 

When ASP.NET was introduced developers knew that things were changing (especially if you came from a classic asp world).  You just weren’t quite sure what you needed to do to make these changes work for you.  I remember the first two years of ASP.NET as a blur of failed projects; trying to find reason to implement the fairy dust of web service based architects (soon to tagged as SOA).   Working with the MVC framework gives me a similar feeling…….I know it’s a needed change but I’m not if Microsoft’s has made the reason we need the MVC framework very clear.

While MVC is a small framework and tiny in comparison to the changes brought on by the Post-Classic ASP cold war, it does hint at what’s to come.  The more I work with Javascript libraries and “Service” based ASP.NET projects the less of a role I see ASP.NET playing in the rendering of UI Controls. That is where MVC pays its dividends,  taking care of the server side architect and letting client based scripting take care of the UI Magic.

MVC plugged into the traditional way of building ASP.NET makes little sense (the least sense for the drag and droppers).  But, if web apps continue to evolve into more Rich State filled Balls of AJAX wonder, MVC may be the next fairy dust.

Here is a link to Alan Lok’s excellent look at MVC and EXTJS: Ext JS and ASP.NET MVC

public partial class CommunityPromotion
{ 
     public bool IsNew
     { 
          get
          { 

               return this.ID <1; 
          } 
     } 

     public bool isDeleted { get; set; } 
} 


//create a partial class for the database context and extend an interface that defines custom methods 


public partial class DatabaseDataContext : IPromotionRepository
{ 

     public void Save(Promotion promotion)
     {
          if (promotion.ID < 1)  // if its a new promotion just insert it
          {
               this.Promotions.InsertOnSubmit(promotion); 
          }
          else  // if its an update to a promotion... check to see if any of the nested objects are marked as new or for delete
          { 

            //  this grabs all of the promotion objects that are new and inserts them 


               this.CommunityPromotions.InsertAllOnSubmit(promotion.CommunityPromotions.Where(a => .IsNew));
               this.CommunityPromotions.DeleteAllOnSubmit(promotion.CommunityPromotions.Where(a => a.isDeleted));
  
          } 

          this.SubmitChanges(); 
     } 

} 

this was pulled from a good article outlining the application development infrastructure.  I often reference this to remind me what each software development environment is responsible for.

.......

Application development, deployment, and hosting in a large IT organization often involve separate development teams working in parallel on numerous applications for deployment to a shared hosting environment. This kind of dynamic environment demands an orderly process for promotion of application elements from the initial development stages to a highly stable and tightly managed production environment.

The ways in which an application is exercised at the various stages of its life cycle and deployment schedule require several different parallel instantiations of the application. These instantiations or environments go by many names in different organizations, but the following names are commonly used:

Straight from the Microsoft "Rapid Application Development" book..

1. People
2. Process
3. Product
4. Technology

                          People

            Process     +     Product

                      Technology

Impacts software productivity and quality

The most effective practices leverage human potential not technology

Areas that effect peopleware:

o   individual ability

o   individual motivation

o   overall team ability

o   overall team motivation

Staffing for team projects: Five principals

o   Job Talent

o   Job matching

o   Career progression

o   Team balance

o   Misfit elimination

o   Other: design ability, language experience, machine and environment experience, and applications area experience

Team Organization

o   Match project size

o   Match product attributes

o   Match schedule goals

Motivation

o   Is your strongest ally on RAD Projects

Management methodologies and technical methodologies

o   neglect to process will cause developers to work inefficiently and at cross purposes when there isn’t a need

Rework avoidance

o   avoid doing things twice (get accurate requirements)

Quality assurance

o   assure that product has acceptable level of quality

o   detect errors at the stage when they are least time consuming to fix

Development fundamentals

o   source control

o   requirements management

o   design

o   construction

o   configuration management

Risk management

o   merge schedule related tasks

Resource targeting

o   effectively focused

o   contribute to overall productivity

o   Best practices: productive offices, time box development, accurate scheduling and voluntary overtime

Lifecycle planning

o   lifecycle models make it easier to identify and organize the many activities of a software project.

Customer orientation

o   put yourself on same same side as customer

o   best practices that help with this: staged releases, evolutionary delivery, evolutionary prototyping, throw away prototyping, and principle negotiation.

Reduce product feature set: reduction in project scheduling

Product size and characteristics offer opportunities to cut development time

I ran across a document about software development from IBM at one point in my past.  I thought I would archive it here.

1. Development process - It is important to choose the appropriate development lifecycle process to the project at hand because all other activities are derived from the process. For most modern software development projects, some kind of spiral-based methodology is used over a waterfall process.

  2. Requirements - Gathering and agreeing on requirements is fundamental to a successful project. This does not necessarily imply that all requirements need to be fixed before any architecture, design, and coding are done, but it is important for the development team to understand what needs to be built.

 3. Architecture - Choosing the appropriate architecture for your application is key. Many times IBM is asked to review a project in trouble and we have found that the development team did not apply well-known industry architecture best practices. A good way to avoid this type of problem is to contact IBM. Our consultants can work side by side with your team and ensure that the projects get started on the right track. Tried and true practices are called patterns and they range from the classic Gang of Four [6] patterns, Java patterns [7], to EJB design patterns [8]. Sun's equivalent is the Core J2EE Patterns catalog [9]. Many projects fail as discussed in the introduction. The study of these failures has given rise to the concept of antipatterns. They are valuable because they provide useful knowledge of what does not work, and why.  

4. Design - Even with a good architecture it is still possible to have a bad design. Many applications are either over-designed or under-designed. The two basic principles here are "Keep it Simple" and information hiding. For many projects, it is important to perform Object-Oriented Analysis and Design using UML. There are many books on UML, but we recommend UML User Guide [11] and Applying UML and Patterns [12]. Reuse is one of the great promises of OO, but it is often unrealized because of the additional effort required to create reusable assets. Code reuse is but one form of reuse and there are other kinds of reuse that can provide better productivity gains. 

5. WebSphere application design - IBM has extensive knowledge of the best practices and design patterns for the WebSphere product family. Each project is different and our consultants have the experience to help you. There is still a tremendous return on investment (ROI) even if you only use the consultants for a short time because you save the costs later in the project. Our experts have also published a great deal of this wisdom, including considerations for high-performance Web sites and guidelines for autonomic computing. 

6. Construction of the code - Construction of the code is a fraction of the total project effort, but it is often the most visible. Other work equally important includes requirements, architecture, analysis, design, and test. In projects with no development process (so-called "code and fix"), these tasks are also happening, but under the guise of programming. A best practice for constructing code includes the daily build and smoke test. Martin Fowler goes one step further and suggests continuous integration that also integrates the concept of unit tests and self-testing code. Note that even though continuous integration and unit tests have gained popularity through XP, you can use these best practices on all types of projects. I recommend using standard frameworks to automate builds and testing, such as Ant and JUnit. 

7. Peer reviews - It is important to review other people's work. Experience has shown that problems are eliminated earlier this way and reviews are as effective or even more effective than testing. Any artifact from the development process is reviewed, including plans, requirements, architecture, design, code, and test cases. Karl Wiegers paper on the Seven Deadly Sins of Software Reviews explains the correct ways to perform peer reviews. Peer reviews are helpful in trying to produce software quality at top speed. 

8. Testing - Testing is not an afterthought or cutback when the schedule gets tight. It is an integral part of software development that needs to be planned. It is also important that testing is done proactively; meaning that test cases are planned before coding starts, and test cases are developed while the application is being designed and coded. There are also a number of testing patterns that have been developed. 

9. Performance testing - Testing is usually the last resort to catch application defects. It is labor intensive and usually only catches coding defects. Architecture and design defects may be missed. One method to catch some architectural defects is to simulate load testing on the application before it is deployed and to deal with performance issues before they become problems. 

10. Configuration management - Configuration management involves knowing the state of all artifacts that make up your system or project, managing the state of those artifacts, and releasing distinct versions of a system. There is more to configuration management than just source control systems, such as Rational Clearcase. There are also best practices and patterns [13] for configuration management. 

11. Quality and defects management - It is important to establish quality priorities and release criteria for the project so that a plan is constructed to help the team achieve quality software. As the project is coded and tested, the defect arrival and fix rate can help measure the maturity of the code. It is important that a defect tracking system is used that is linked to the source control management system. For example, projects using Rational ClearCase may also use Rational ClearQuest. By using defect tracking, it is possible to gauge when a project is ready to release.

 12. Deployment - Deployment is the final stage of releasing an application for users. If you get this far in your project - congratulations! However, there are still things that can go wrong. You need to plan for deployment and you can use a deployment checklist on the Construx Web site. 

13. System operations and support - Without the operations department, you cannot deploy and support a new application. The support area is a vital factor to respond and resolve user problems. To ease the flow of problems, the support problem database is hooked into the application defect tracking system. 

14. Data migration - Most applications are not brand new, but are enhancements or rewrites of existing applications. Data migration from the existing data sources is usually a major project by itself. This is not a project for your junior programmers. It is as important as the new application. Usually the new application has better business rules and expects higher quality data. Improving the quality of data is a complex subject outside the scope of this article. 

15. Project management - Project management is key to a successful project. Many of the other best practice areas described in this article are related to project management and a good project manager is already aware of the existence of these best practices. Our recommended bible for project management is Rapid Development by Steve McConnell [14]. Given the number of other checklists and tip sheets for project management, it is surprising how many project managers are not aware of them and do not apply lessons learned from previous projects, such as: "if you fail to plan, you plan to fail." One way to manage a difficult project is through timeboxing.  

16. Measuring success - You can measure your development process against an industry standard known as the Capability Maturity Model (CMM) from the Software Engineering Institute at Carnegie Mellon University. Most projects are at level 1 (initial). If you implement the best practices described above and the guidelines in the companion article, Guide to Running Software Development Projects, then you could be well on the way to achieving a higher maturity level and a successful project.

1. Seeing Waste
2. Value Stream Mapping
3. Feedback
4. Iterations
5. Synchronization
6. Set-Based Development
7. Options Thinking
8. Last Responsible Moment
9. Making Decisions
10. Pull Systems
11. Queuing Theory
12. Cost of Delay
13. Self Determination
14. Motivation
15. Leadership
16. Expertise
17. Perceived Integrity
18. Conceptual Integrity
19. Testing
20. Refactoring
21. Measurements
22. Contracts
 
If something does not directly add value as perceived by the customer, it is waste. If there is a way to do without it, it is waste.

"First and foremost, our mission is to provide customer value."

I had read these ideas somewhere but not sure where.  I give credit to whoever wrote them. 

........

The success of a software project depends on not writing source code too early in the project.

You can't trade defect count for cost or schedule unless youre working on life critical systems. focus on defect count; cost and schedule will follow.

Silver bullets are hazardous to a projects health though software industry history suggests that vendors will continue to claim otherwise.

Half hearted process improvements is an especially damaging kind of silver bullet because it undermines future improvement attempts.

Despite its name, software isn't soft, unless its made that way in the first place, and making it soft is expensive.