Generate the build script for test

The suggested practice is to have 2 separate projects for android, one the source and the other for the tests. The following command will generate a build.xml for the test project. Replace the with the path of the source path.

android update test-project -m ../<project-path> -p .

One problem I’ve seen is that, it does not break the build even if there are failures in the test. Issue is reported here:

http://code.google.com/p/android/issues/detail?id=14241

I had to override the run-tests target as mentioned below to fix this issue:

<target name="run-tests" depends="-install-tested-project, install"
description="Runs tests from the package defined in test.package property">
    <echo>Running tests ...</echo>
    <exec executable="${adb}" failonerror="true" outputproperty="tests.output">
        <arg value="shell" />
        <arg value="am" />
        <arg value="instrument" />
        <arg value="-w" />
        <arg value="-e" />
        <arg value="coverage" />
        <arg value="@{emma.enabled}" />
        <arg value="${manifest.package}/${test.runner}" />
    </exec>
    <echo message="${tests.output}"/>
    <fail message="Tests failed!!!">
        <condition>
            <or>
            <contains string="${tests.output}" substring="Error" />
            <contains string="${tests.output}" substring="Fail" />
            </or>
        </condition>
     </fail>
</target>

You can change the ant commands to clean run-tests release in Jenkins to run the tests as part of packaging.

Next I will be writing about how to start emulator from Jenkins while running the tests.

Image via Wikipedia

We constantly try to provide accurate estimates that we can defend with confidence. But there are situations where we end up making mistakes. And this post describes one such situation.

One of our clients asked us to estimate a feature. As usual, we sent back an estimate without much delay.

After few days, our client asked us to implement this feature. When we started to think about implementing the feature we found that it would take at least double the time that we’d earlier estimated. We’d put ourselves in a bad situation. It would of course be very hard to convince the client as to why there was this much deviation – considering we discovered this even before actually starting to implement the feature. We did the 5 why’s to get to the root of the problem.

We discovered the following reasons:

• We hadn’t gone through the usual process of breaking down the feature to the desired level of granularity. Digging deeper, the following root causes emerged:
  • The project had been on ‘pause’ mode for a couple of weeks and we had gotten busy with other things in the meanwhile.
  • The value of the feature [to the end user] was not completely obvious to us.

The solution? We’ve decided to ask ourselves the following question before sending across an estimate to any client in the future:

Is this a confident estimate? Can we defend the estimate with proper reasoning?

The above will force us to think again about the estimate and help us become more consistent.

Oh, and one more thing. Ask the stakeholder as to what value the feature is going to add – if it is not obvious. Don’t assume that you’re right!

If you’re curious on what the client’s reaction was, when we sent across the revised [and much larger] estimate, ask us in the comment below We’d love to hear from you!

We’ve set up our CI server for building android apps. We use Jenkins as our CI server, but the same steps can be applied to any CI server.

Setup Android Environment on CI server

You need to first install the android SDK and platform tools on the CI server. The steps are well defined here. You can run the command android update sdk --no-ui if the CI server is in an headless environment.

Generate Build script

Using android SDK tool , you can generate build script for the android project which contains the standard steps for building the app such as clean, compile, release, install etc. The following command will generate the build script, replace <appname>, <target> and <project path> accordingly.

android update project -n <appname> -t <target> -p <project directory>

This will create build.xml file under the project directory. You need to create build.properties file with the following contents:

key.store=path-to-keystore
key.alias=[alias]
key.store.password=[pw]
key.alias.password=[pw2]

You can generate the key file using keytoool or you can generate the key file from eclipse. Run the command  ant clean release, which will compile the files, and generate the apk files (it generates signed, unsigned and unaligned files). The signed version can be used for uploading to Android Market or for installing directly on any device. Couple of stuff to be noted here are:

• Ant version has to be 1.8.0 or higher.
• Put the external libraries in the libs directory. Build script automatically picks up the libraries put under libs directory, otherwise the script need to be changed to look at a different classpath.

Checkin the build.xml, build.properties and the key file into the repository.

Setup the CI server

The CI server has to run the ant script for building the app. One more setting what we’ve done in our Jenkins server was to archive the apks as artifacts (available in the post build action). In upcoming posts, I will cover how to do the following:

1. Running android tests
2. Running code/test coverage tools
3. Actual deployment

References: http://skaug.no/ingvald/2011/09/android-app-with-jenkins/

• All the migrations have the down method defined properly. You can check for this by running rake db:migrate:redo
• The schema.rb exists in the repository. This is one of the suggested practices for Rails.

The script is very simple, it extracts the version from the schema.rb and runs the rake db:migrate with the same version. The task gets run automatically along with deploy:rollback. This approach should work for most small and medium complexity Rails apps.

Continued..

• Continuous Delivery – Part 1: Our Jenkins Build Pipeline setup
• Continuous Delivery – Part 2: Code metrics with metrical
• Continuous Delivery – Part 3: Running custom rake tasks during deployment

• Reindex the solr/lucene indexes if any new field has been added to the index
• Some custom rake tasks , say for eg: task to update values in the DB, which you don’t want to add to migrations

We do have Cap tasks for running the required rake tasks, but again it was done manually. So we’ve to remember to run them after deploying to production.  And also there is no way to track what steps were followed for a certain deployment.

I used the build parameter again for fixing the above issue i.e. after accepting the extra tasks as parameter for the build for the first job, they will be passed on to the downstream jobs. In this way the same deployment steps will be automatically followed for rest of the jobs in the pipeline.

The only difference in this case is that – sometimes the parameter can be empty. This check has to be done in the scripts i.e. if the value exists for the parameter, execute the extra tasks along with the normal deployment else just do a normal deployment.

Continued..

• Continuous Delivery – Part 1: Our Jenkins Build Pipeline setup
• Continuous Delivery – Part 2: Code metrics with metrical
• Continuous Delivery – Part 4: Rolling back database migrations with Capistrano rollback

The steps I followed for setting it up in our Jenkins server are:

• Install the gem. I installed it under our ruby 1.9.2 in RVM.
• Create a .metrics file under your projects directory with the settings. This is not mandatory, but I had to use this because some configurations do not seem to be working with ruby 1.9.2. I’ve mentioned those below along with the contents of the .metrics file
• Create a job in Jenkins server, mention the repository details and for build step give the shell command as rvm 1.9.2 -S metrical
• Configure the HTML Publisher plugin for showing the nice Graphs generated by metric_fu as part of the build report. The default report location is tmp/metric_fu/output under the project directory. You can change the same in .metrics file.

As I mentioned above, tools such as flay and flog, which comes as part of metric_fu, have issues working with ruby 1.9.2. And the same case with stats graph and syntax highlighting. The same case with rcov also. So I had to create a .metrics file with the following contents and put in the project dir:

MetricFu::Configuration.run do |config|
        config.code_dirs = ['app', 'lib']
        config.syntax_highlighting = false
        config.metrics  = [:churn,:reek,:roodi,:hotspots,:rails_best_practices]
        config.graphs   = [:reek, :roodi, :rails_best_practices]
        config.rcov[:test_files] = ['spec/**/*_spec.rb']
        config.rcov[:rcov_opts] << "-Ispec" # Needed to find spec_helper
end

For test coverage I’ve used Simplecov which is easy to setup. It will generate the coverage report whenever you run the tests. This also generated html report which can be integrated easily into Jenkins. As mentioned here in the issue list, it does not generate the report when you are running with spork.

Continued..

• Continuous Delivery – Part 1: Our Jenkins Build Pipeline setup
• Continuous Delivery – Part 3: Running custom rake tasks during deployment
• Continuous Delivery – Part 4: Rolling back database migrations with Capistrano rollback

The pipeline we have now is very simple. They are:

• Run all the RSpec tests
• Run the Javascript Unit tests
• Deploy to staging
• Deploy to production

The problems we had with our earlier approach, and how the current approach solved those problems:

What we had in the past was, simple scripts with the above items. We had a job for running the tests. If the build was successful, we would run capistrano scripts to deploy to staging and once the customer gave the thumbs up, we’d push it to production.  There are a series of steps that we had to do for each deployment. For eg:  for staging deployment:

• Merge the “staging” branch with master
• Run the cap script
• Tag the build

And almost similar steps needed to be followed for production deployment also. The usual problems that we used to face are:

• There is no way we knew what deployment was done on a certain day for what
• All staging updates would not get pushed to production, so it was a tough task to identify which tag to merge when we were finally ready for a production update

With the current approach, the advantages are:

• Its a one click deployment, so rare chance of making any kind of mistake
• We can clearly see when a feature has been moved to staging, and whether it has been pushed to production or not
• Anyone can do deployment as it is just a click of a button. We may even be able to teach the client how to do this.

One of the major problems I had faced in the earlier “manual” system – was the inability to pass parameters between builds. For example, once the app would be moved to staging, a tag is created and when its ready to be pushed to production, we’d have to merge that particular tag to production and then push it.

To achieve the above I used the “parameterized build” option. In Jenkins, the parameters get converted to Environment Variables and got passed to the downstream job. This is how you can set the parameterized build:

When you fire the build, it prompts for the value of the TAG parameter. Once entered this would be available as an Environment variable for this job and all its downstream jobs.

For eg: in the pipeline, the downstream project is “Stage deploy”, and it will know the value for TAG and uses that value to create a tag in Git after successful deployment.

When it is ready for production deployment it merges with the same tag and deploys to production. In the latest version of the plugin, moving to the next step is not triggered automatically and you can retry the failed ones. The following is a sample pipeline dashboard view:

Note: Capturing the parameter will happen only for the first job. So, if the parameterized job is not the first one in the pipeline, the capture of the parameter will not happen.

So we’ve reached the “one click deployment stage” with this. But we’ve still got the following pending tasks – based on what’s mention in the book  Continuous Delivery:

• Rolling back the bad builds
• Add non-functional tests to the pipeline i.e. performance and security tests
• Feature flags
• Zero downtime deployment
• Canary Releases (Something on the lines of A/B testing)

Continued..

• Continuous Delivery – Part 2: Code metrics with metrical
• Continuous Delivery – Part 3: Running custom rake tasks during deployment
• Continuous Delivery – Part 4: Rolling back database migrations with Capistrano rollback

References:

• http://weblogs.java.net/blog/johnsmart/archive/2011/03/10/build-pipelines-jenkinshudson
• http://continuousdelivery.com/patterns
• http://www.quora.com/How-does-Etsy-manage-development-and-operations

After the launch, we realized that the number of issues reported over the last few weeks of the project were much higher than what we were used to for most of the project. So not only did we have a tired team at launch time, we also had lower quality. In the retrospective that soon followed the launch, we discussed this in great detail and came up with one important conclusion:

We were missing the forest for trees – by focusing  too much on the individual practices, but not appreciating the overall XP Process. Had we been perhaps more aware of the value we were getting by following the practices, we’d probably tried to find or create new practices that would have suited the changing context better.

Thankfully though, I got this thought that writing to James Shore directly might help as well. James is, among many other things, the author of our favorite book – The Art of Agile Development. What follows is the question/answer emails I exchanged with him:

My Question to James:

We are a team that has transitioned to XP for a while now. We have been following almost everything mentioned in your book “The Art of Agile Development”. We found that the project was more stable than any other project that we’d done in the past.

But we found recently that we were not able to sustain the pace with which we had started the project. There was this “Crunch Mode” in the last few weeks of the project with everyone in the team putting in long hours of work.

We really want to avoid to such situations in our future projects. Can you suggest as to how we could achieve that?

And the response I got from him:

Glad to hear it worked for you! It’s hard to know what happened since I wasn’t there, but there are several possibilities:

- You might have overcommitted. In that case, take a second look at the “Release Planning” and “Risk Management” practices. You might have been better off reducing scope or delaying the release date.

- You might have had incurred too much technical debt. In that case, remember the maxim that you should always leave the code better than you found it, and take a second look at the Slack, TDD, Refactoring, Simple Design, and Incremental Design practices.

- You might have had too many bugs. In that case, take a second look at the “No Bugs” practice and remember that you need to keep things clean and bug-free all the time rather than saving up bug-fixing for the end. Also consider doing the “Exploratory Testing ^[1]” practice from the beginning of the project, and not saving testing for the end.

- You might have had a difficult relationship with your key stakeholders. This problem, in particular, can result in excessive schedule pressure, which will lead to all of the other problems I’ve mentioned. In that case, see the “Trust” practice and enlist the help of some politically-savvy people in your organization. You might also be interested in the Rabu project I’ve started (http://www.teamrabu.com).

Finally, consider that a week or two of crunch mode at the end of a longer project isn’t always a bad thing, so long as that’s all it is.

Best wishes,

Jim

I guess our problem was that we had overcommitted. We also did not consider practices like the Risk Management and Slack.

Now speaking of action items, we’ve come up with the following:

• Commit to deliver features only after following the Risk Management techniques mentioned in the book
• Introduce Slack as part of every iteration
• And finally, to ensure we don’t forget to appreciate the process enough, we’ve added one additional item in our Navigator’s^[2] checklist.
  • [Update] The value of this action item is debatable. Its been about 3 weeks since we put it up on the checklist – but we’ve got the tendency to ignore anything that doesn’t change frequently enough. So we’ll need to come up with a better solution for the “lack of appreciation” problem. Any ideas? Please help us by commenting below. Thanks!

Notes:

[1] This practice is yet to be put online.

[2] This refers to the role of the Navigator in the context of Pair Programming

There were quite a few presentations this year which I felt were very useful for me. Those are:

• Ruby Plus Rails Plus Your Application Minus Rails by Brian Guthrie
• Test Load Balancer: Rocket Booster for your Build by Janmejay Singh and Pavan
• Continuous Delivery in Ruby by Srushti Ambekallu and Brian Guthrie

And its needless to say that the keynotes by Yehuda Katz, Chad Fowler, Nick Sieger and Ola Bini were awesome too. Especially Chad Fowler’s session gave a new perspective on “Service” and who should be considered as a “Customer”. And Nick Sieger’s closing note gave a different perspective on how to contribute back to the community by conducting workshops and with Kidsruby.

So whats next? Yes, implement the stuff we learned. So these are the immediate action items for us:

• More automation for the entire release mechanism. We do have a CI server and do have a Cap script for deployment. But we do not have a “one click deployment” process. A “Build pipeline” plugin for Hudson should help us achieve that.
• Tools like Chef/Puppet for server configuration management. We’ve tried rubber, but are yet to use it for any production setup.
• Use TLB to run tests in parallel.  Setup seem to be pretty straightforward. This will be a huge help when you are doing many deployment in a day.

Update: One talk I missed in the list was Designing High Throughput Web Service Clients by Sherin. He was able to convey the problems he faced and how he went ahead and solved those.

Venkat is the founder of Agile Developer and the author of several books. Consultant and Adjunct Professor are some of the other hats he wears. It was an inspiring experience. While we were reassured that our current practices are on the right track during most of the talk, there were a some things which caused us to rethink the way we do things.

Why do we need Code Quality?

One of the first things Venkat asked us to think about was why code quality was necessary in the first place. Most of the audience had reasons like ‘Easy Maintainence’, ‘To make a better product’, ‘Company’s image’ etc. These were reasons which we had on the top of our mind as well. However his reasoning was quite different:

“We need Code Quality to be truly Agile“

What this means is that rapidly adding new features or quickly changing existing features to incorporate feedback is not possible without clean code.

We have been focusing on writing cleaner code for quite some time as well. But our approach to improving in this area has been limited to reading books and online articles about better approaches to write clean code and internal discussions with our team. One essential ingredient that we were missing was using an existing sample, something near perfection which we could look at and learn from or at least be inspired by.

Pair Programming Vs Code Review

Code Review aims to provide external feedback. But we already do pair programming which tries to achieve nearly the same. The danger lies in the navigator becoming so invloved that he is not externalized enough. So the solution is to switch pairs every half a day.

Additional Action Items

After having a brainstroming session with the entire team here, we decided to include the following suggestions from Venkat’s talk to our existing practices:

Starting “Perspective” based Peer Reviews
This is mandatory for  projects which have only one pair (and hence cannot switch pairs). The reviewer should have enough familiarity(perspective) with the project to be able to make a judgement on the design and quality. This is also useful for projects which can switch pairs – but not mandatory.

Dedicated time to pay off technical debt
While our iterations always have time baked in, to clear technical debt, we often spend that time doing something else instead. More disclipine in paying down our technical debt is something we intend to work on.

Code Quality metrics in the Informative Workspace
While ‘name and shame’ is certainly not something we do, Venkat’s belief in publicising quality metrics as a way to encourage code quality [in our informative workspace] is something we will explore in the coming weeks.

]]> http://www.multunus.com/2011/04/venkat-subramaniam-code-quality/feed/ 0