Some selenium tips

UPDATES

• May 31, 2012: Added the tip to capture the screen when test fails.
• May 11, 2012: Added the tip that about keeping the browser window displaying (or hidden) during the tests.
• May 11, 2012: Rephrase the scroll to a button before clicking it tip.

Watch out the outdated articles on the internet.

Selenium 2.0 is completely different from Selenium 1.x. Selenium 2.0 is also called the selenium webdriver. So always add the keyword webdriver when googling for answers to your selenium related questions.

Implement the web UI in a modular way so it's more selenium testable.

Modularize your view logic so that you only update the part of DOM that is needed to change when your models change. If you tend to re-create a bigger part of the DOM than necessary, it's not only a waste but also could introduce risk to your functional tests written in Selenium.

Reduce unnecessary dependency on DOM structure, make element locating logic as simple as possible.

When you need to locate an element, try not rely on the DOM structure too much - for example, using code logic to locate element is the most risky one. The best approach is probably to always use a scoped CSS selector with 1 or 2 levels of IDs, And if you can locate it in one selector, don't do it in two. For example

  label = driver.find_element("#info-panel #name-label")

is more robust than

  panel = driver.find_element("#info-panel")
  label = panel.find_element("#name-label")

Do waiting in selenium the smart way.

Don't use implicit wait blindly. Implicit wait makes sense when you use find_element to find one element. But when you try to locate multiple elements by driver.find_elements, the driver will always wait the whole timeout period if implicit wait is set. That might not be what you always want. I usually write my own safe find_element method. Here is an example in the base class of my page objects:

    def s selector
      wait_until { @driver.find_element css: selector }
    end

    def wait_until(&block)
      wait = Selenium::WebDriver::Wait.new(timeout: 10, interval: INTERVAL)
      wait.until &block
    end

So that I can write the following code in my page object

   def submit_order
     s('button#submit').click
   end

The short method name "s" is inspired by jQuery. Here it will keep polling the DOM for 10 seconds until it finds the button with id "submit". It's like implicit wait but only for finding one element. When you really need to wait for multiple elements, you can use an explicit wait, which, to me, makes more sense than a hidden implicit one.

Set the initial browser window size when using Chromedriver.

Ruby code:

  profile = Selenium::WebDriver::Chrome::Profile.new
  profile['browser.window_placement.top'] = 0
  profile['browser.window_placement.left'] = 0
  profile['browser.window_placement.right'] = 1024
  profile['browser.window_placement.bottom'] = 768
  driver = Selenium::WebDriver.for :chrome, profile: profile

This works in both Windows and OSX (will try Linux and update here)
Bad news for Java, C# and Python coders though, it seems that as of now setting chrome preference is not supported in the java version of Webdrive. Your best chance could be creating a ChromeProfile class based on the exiting FirefoxProfile class.

Scroll to a button before clicking it.

Clicking buttons sometimes randomly fail. It could be caused by the fact that the button has to be the view area to be clickable and somehow the selenium auto scroll failed. In this case, add a scroll to button will improve the robustness of your suite.

When running the test using Firefox, it matters whether the browser window is displayed on the screen or hidden behind other windows.

From my experience, my guess is that selenium interacts with the browser in slightly different ways depending on if the browser is displaying in the front or not. There is rare case that certain selenium operations only work when the Firefox is displaying in the front. When running the selenium suite, changing the z position of the browser window (and thus either show or hide the browser window from other applications) can affect the tests. So you get more consistent results by keeping the browser either showing in the front(or hidden in the back) during the course of full suite.

Capture screenshots when test fails (RSpec)

In your spec_helper.rb

  RSpec.configure do |config|
    config.after(:each) do
      capture_screen_when_fails(example, @page) 
    end

    def capture_screen_when_fails example, page
      if(example.exception.present? and page.present?)
        page.capture_screen(example.description) 
      end
    end
  end

Note here that you need to keep your page object instance in an instance variable(in my case @page) in your spec. I used a naive way to name the screenshot after the example's description.
Now in your base class for your page object.

    def capture_screen filename
      path = "PATH_TO_TMP/#{filename}.png"
      @driver.save_screenshot(path) 
    end

Note that @driver is the instance variable holding the selenium webdriver currently running.

Test first != TDD

Recently I've seen a misunderstanding of TDD(test driven development), that is, if you always write test first and write code that passes that test, and voilà, you are TDDing, everything will be better now. That is not TDD. That is 100%-test-coverage-driven-development.

TDD is NOT about achieving 100% test coverage (which IMO is meaningless).

TDD is about using test to drive development.

More specifically, TDD requires three equally important types of activities:

• writing test
• writing the simplest code that passes the test
• refactoring code that keeps tests passing

A common misunderstanding is that the refactoring step is optional. It's not. The following is not TDD.
First the test:

   describe 'alert' do
      it 'should log the message' do 
         logger.should_receive(:info).with('a message');
         subject.alert('a message');
      end
   end

Then the method in tested class:

    def alert(msg)
       logger.info(msg)
    end

Then you are done with this method. This is NOT TDD.
First, you didn't refactor the code. Second, it's not likely that the test will help when you need to refactor this code in the future. In one sentence, the refactor-and-keep-tests-green part is missing.
So what will a TDDer do here? Simply write the code without the test. A typical argument against it is that what if some other developer accidentally delete the logger.info(msg) code? Well, that's not the purpose of TDD, that is what version control is for.

In fact, it can be argued that this test is a form of code duplication. It's just that this duplication is very easy to detect (if you change one place without the other, the test will break.)
Things could be a bit different if your code is like this.

   describe 'alert' do
      it 'should log the title and message' do 
         logger.should_receive(:info).with('a title: a message');
         subject.alert('a title', 'a message');
      end
   end

   def alert(title, msg)
      logger.info(title + ': ' + msg)
   end

There is a possibility that you need to refactor to improve the performance of the string concatenation. In this case, the test here has some value.

So, the purpose of tests in TDD is to facilitate refactoring.

In many cases, tests give you the ability to refactor code with the confidence that it won't break functionality. In other cases, you may have to change some tests to do your refactoring, because unless it's a real black box end to end test, you are always going to have some form of logic duplication in your tests.
So a test is probably helpful for some refactoring but impeding for some other refactoring.

The value of a test is determined by the likelihood of it helping refactoring v.s. the likelihood of it impeding it.

When writing tests in TDD, we need to maximize this value with as less cost as possible. Tests on constructors, accessors, constants and simply delegations are typical examples of test with zero or negative value. Pure interaction tests are likely to have less value than a more end-to-end test but it should be evaluated on case-by-case basis.
You also need to take cost into consideration - for example, end-to-end test might takes more effort to write and/or more resource to run.
There is no silver bullet. Always-write-test-first isn't one. It takes some effort to determine when to write tests and how to do so, but it's not too hard - just focus on the value of the test, that is, how much benefits it can bring to immediate and foreseeable refactoring.

Update on Feb 11, 2012

I got the chance to discuss with Martin Fowler about this topic last night. The conversation started when I said "some people think TDD is all about write test first and implement it then done." Martin immediately replied "no, no, no, always refactor later. It should always be red, green, green." Then he quickly reaffirmed my understanding of TDD addressed above:
1, Achieving 100% of test coverage is NEVER the goal of TDD. A sound value of test coverage in TDD should be in the 90's.
2, Tests are for the sake of refactoring. They are written for testing "interesting" stuff. He would not test simple delegation (like the example given above) . Testing a simple delegation using mock has no value since it does not provide any benefits to future refactoring.
He also preferred test at a higher level which I couldn't agree with more. Recently I found that the higher level your tests are the more value they provides. Lower level unit tests are cheaper, but you need to keep in mind the value they can bring. Again, tests are for the purpose of confirming that some "interesting stuff" works, not that some code were written.

Testing private methods in RSpec

Why testing private methods? Well, it's not really about private vs public, if you want really fine granular unit tests that always only test no more than a couple of lines of code, you will need to do partial mocking and test private methods.

I heard argument that testing private methods exposes too much implementation and thus makes later refactoring harder. My argument is that unit test is part of the implementation. Fine grained "real unit" tests is very easy to read and understand. They help clarify the intent of that couple of lines of code in your target class. If you change implementation code, it should be perfect normal if you also need to change that simple unit test. On the other hand if your tests are in a larger granularity, then in each test, either you test a lot of code or your use a lot of mocking. Either case, chances are whenever you change implementation, you would need to change even more test code.

Another common practice is to extract private methods to another class and make them public and test from there. To me, there are only a few valid reasons to introduce a new class (or in general, to design), being able to test private methods isn't one of them.

Alright, with excuses all said (your argument is welcome), here is how I test private methods in ruby with rspec. I defined a global method in a file called describe_internally.rb in my test folder

def describe_internally *args, &block
    example = describe *args, &block
    klass = args[0]
    if klass.is_a? Class
        saved_private_instance_methods = klass.private_instance_methods
        example.before do
             klass.class_eval { public *saved_private_instance_methods }
        end
        example.after do
             klass.class_eval { private *saved_private_instance_methods }
        end
    end
end

then whenever I need to test private methods for a class (say Foo), I use "describe_internally Foo" instead of "describe Foo". If you prefer, you can organize these two types of tests in the same file as the below example (say Foo is a class with two methods-a public one: "kick" and a private one: "aim" which returns the target to be kicked)

describe Foo do
   describe "kick" do
     it "should kick at where aim is" do 
        foo = Foo.new         
        foo.should_receive(:aim).with(steve_jobs).and_return :a_place
        foo.kick steve_jobs
        steve_jobs.should be_kicked_at :a_place 
     end
   end
end
describe_internally Foo do
   describe "aim" do 
      it "should aim at where the butt is" do
           Foo.new.aim(steve_jobs).should be steve_jobs.butt
      end    
   end
end

(Although in this case, I would just put all specs under the describe_internally block, because the first spec also requires knowledge of the private method and from my experience there is seldom any problem caused by that)
If you need an even more fine control of the scope of where you want private methods exposed, Jay Fields wrote a blog long ago giving another approach to achieve it http://blog.jayfields.com/2007/11/ruby-testing-private-methods.html

Test Driven Development on Android

Although at the very beginning, TDD, especially the UI part, looked challenging on Android, now after about 90 tests and 4,5 epic stories, I am comfortably to say that Android app can be developed in a TDD fashion reasonably smooth. You just need to accept that you can't test the interaction between UI and domain model using mock framework. Then you design in a way so that such need will be reduced to minimum. Some Android API might not be test friendly, you can solve that by introducing some middle layer abstraction.