Using Docker to construct a Selenium Grid

When thinking about possible uses for containerisation, one which jumped out at me was web testing. This normally involves large numbers of big, slow and fragile VMs.

I started with making a simple PhantomJS docker container, which removes the necessity of installing PhantomJS on your machine (which can be annoying complex). The image is available on Docker Hub here.

The dream would be using containers to construct a large Selenium grid of different browsers and versions, taking advantage of the performance/reduced size benefits of containers to be able to run a bigger/faster grid than you normally would with VMs. As the containers would be disposable, you could easily start/stop Selenium nodes for browsers when needed (i.e. a set for quick smoke tests and one for more extensive overnight tests). Developers would be able to use the same set of containers and hub scripts, so wouldn’t have to mess around with VMs and scripts to run tests to validate and reproduce issues, making it easier to find issues early.

The main problem is containerisation is currently limited to Linux OS containers only, which means no testing on OS X or Windows, so no Safari and Internet Explorer (who needs to test that, right). These holes can be filled with VMs spun up by Vagrant and added to the Selenium hub, but it’s not an ideal solution. Still, I believe this is a potential solution for small projects which can’t afford Browserstack or Saucelabs licences for testing.

Below are the Docker commands and Selenium tests to call various browsers:

# Run hub
docker run -d -p 4444:4444 --name selenium-hub selenium/hub:2.46.0

# Run browser containers
docker run -d -P --link selenium-hub:hub selenium/node-chrome:2.46.0
docker run -d -P --link selenium-hub:hub selenium/node-firefox:2.46.0
docker run -d --link selenium-hub:hub stevenalexander/node-phantomjs:2.46.0
# Add vagrant VM browser nodes here

With these commands you now have a Selenium Grid running with a number of browser nodes registered and can run tests against the hub using the RemoteWebDriver targeting individual browser/versions. You can modify the containers used to create containers running different versions or add VMs to register IE/Safari/Mobile nodes.

Useful links:

Advertisements

Docker nginx/dropwizard with Travis CI

Source

Update: Can’t recommend using Travis for this kind of CI, it turned out very flakey once I used actual images rather than the simple echo tool images. Builds would sometimes timeout after 15mins of hanging, either on building images or attaching. Not sure why, but I imagine Travis isn’t really intended for loading multiple containers ~400meg in size (JVM+deps).

Example using Docker to create an nginx image and dropwizard image, then link them together so nginx acts as a reverse proxy for Dropwizard. Can be extended to link together multiple Dropwizard applications. Uses Docker Compose to create and configure the images.

terminal gif

Requires:

To run locally:

gradle run
# ./go

To run containers:

gradle buildJar
docker-compose up -d

# retrieve your docker host IP from boot2docker
boot2docker ip

# curl dropwizard/nginx containers using docker host IP
curl http://192.168.59.103:8080/hello
curl http://192.168.59.103:8090/hello

Details

The docker-compose.yml file configures the two images, creating a dropwizard container and linking it to an nginx container. With the link in place, docker creates a hosts entry for the dropwizard container which can be used in the nginx config volumes-nginx-conf.d/default.conf when setting up the reverse proxy.

Based Travis build on moul/travis-docker.

Using a client image to test the nginx/dropwizard images, as you cannot curl directly from Travis CI. Ideally, once Travis has started the containers in demonised form I would run a test script which uses curl/selenium to test the various endpoints exposed from nginx and hit dropwizard. If this needs to be done via the client then the results of the test can be output to a write-enabled volume and parsed to determine build result, as docker-compose will always return exit code 0 if the containers run.

Automated testing of web applications with complex web service integration points

I’m currently on a project which has a testing problem. The problem is it has a complex web application with multiple web service integration points, some of which are sometimes flaky, have complex dataloads or difficult to setup. This means that our automated web tests (done in Selenium) can be slow, unreliable and tricky to setup (requiring automating setup steps as well as the tests themselves).

Some of those are problems which need to be solved in the service dependencies and can’t be ignored as part of proper integration tests, but at the same time it would be good to be able to quickly test the web UI aspect in isolation for Javascript components (using AngularJS) and browser compatibility without having these separate integration tests wrecking entire test runs.

To achieve this I would like to create a simple web application that mocks and mimics various services, allowing the automated tests to setup canned responses from the services, similar to how a mocked service would be used in a unit test. Basically before an automated test executes it would post one or more canned responses which would go into a stack on the mimicking  service, then as the automated test executes it’s steps should cause requests to fire against the mimic service which will respond from it’s response stack (FIFO). Responses could be added to various queues to mimic different endpoints, resources or services.

Objectives for mimic application:

  • Simple, must be easy to write a canned response or series of responses for tests
  • Fast, if it can’t respond significantly faster than the real integration points theres no advantage
  • Flexible, to handle mocking different types of services
  • Handle multiple endpoints, so you don’t need to host multiple versions for different services

Disadvantages of this approach:

  • Adds complexity to web tests (hopefully balanced out by removing more setup code)
  • Won’t catch regression issues from integration point changes
  • Need to maintain canned responses in tests

Will update when I have some simple code ready on Github.

Dropwizard custom security provider for authentication

Sample application showing how to use security providers in your Dropwizard application.

Includes an example custom security provider which can be injected into your service and used in Resource methods to authenticate requests and apply authorisation. Can be extended to implement a full authentication and authorisation for the service.

Introduction

The majority of the provider in this sample project was based off the BasicAuthProvider from Dropwizard auth library, I wanted to do this to understand better how the injectable security provider works and figure out how it could be used to implement authentication and authorisation.

Also wanted to ensure I could easily extend and test any resources using the security, as the overhead for a security system is important in keeping your code clean and not obscuring the intended purpose with security code.

Details

Implementing the custom security provider requires three classes (in com.example.security):

  • ExampleCredentialsHolds the credentials extracted from the request, e.g. username/password or a token from a cookie.
  • ExampleAuthenticatorTakes the credentials and authenticates them, returning a principle (user) object, throwing an AuthenticationException if the credentials are invalid.
  • ExampleSecurityProviderIs injected into the service so any request decorated with the Dropwizard Auth attribute will be handled by this provider. Extracts credentials from requests, uses an authenticator to check them and throws WebExceptions if not authenticated.
    /**
     * An example security provider that will look at each request when received by an endpoint using the auth attribute
     * and check that it has a header value containing a token and will authenticate the token to get the Principle (User)
     * for the request (otherwise throw an AuthenticationException). That Principle is the authenticated User associated
     * with the request and the resource method handling the request can use it to check authorisation to perform actions.
     *
     * @param <T> The Principle class (User) to be returned when a request is authenticated
     */
    public class ExampleSecurityProvider<T> implements InjectableProvider <Auth, Parameter> {
        ...
        public ExampleSecurityProvider(Authenticator<ExampleCredentials, T> authenticator) {
            this.authenticator = authenticator;
        }
        ...
        private static class ExampleSecurityInjectable<T> extends AbstractHttpContextInjectable<T> {
            ...
            @Override
            public T getValue(HttpContext c) {
                // This is where the credentials are extracted from the request
                final String header = c.getRequest().getHeaderValue(CUSTOM_HEADER);
                try {
                    if (header != null) {
                        final Optional<T> result = authenticator.authenticate(new ExampleCredentials(header));
                        if (result.isPresent()) {
                            return result.get();
                        }
                    }
                } catch (AuthenticationException e) {
                    throw new WebApplicationException(Response.Status.UNAUTHORIZED);
                }
    
                if (required) {
                    throw new WebApplicationException(Response.Status.UNAUTHORIZED);
                }
    
                return null;
            }
        }
    

The security provider is injected into the service in the Service class.

public class ExampleService extends com.yammer.dropwizard.Service<ExampleConfiguration> {
    ...
    @Override
    public void run(ExampleConfiguration configuration, Environment environment) throws Exception {
        environment.addResource(new UserResource());

        // Adds security provider so resource methods decorated with auth attribute will use this authenticator
        environment.addProvider(new ExampleSecurityProvider<User>(new ExampleAuthenticator()));
    }

Resources using authentication just add the Auth attribute to their method signature.

@Path("/user")
@Consumes({MediaType.APPLICATION_JSON})
@Produces({MediaType.APPLICATION_JSON})
public class UserResource {
    ...
    /*
    * Using the Auth attribute will use the injected provider to authenticate all requests to this path
    * You can also use the principal to apply authorisation in code dynamically
     */
    @GET
    public List<User> getAll(@Auth User principal){

        if (!principal.getDisplayRole().equals(User.ROLE_ADMIN)) {
            throw new WebApplicationException(Response.Status.UNAUTHORIZED);
        }
        ...

Testing the resources requires injecting a provider into the test service (note, I used the full one but for simplicity you can inject a mock which would return a controllable principal).

public class UserResourceTests extends ResourceTest {
    ...
    @Override
    protected void setUpResources() {
        addResource(new UserResource());

        // Need to add SecurityProvider to all resource tests for resources using Auth, or you get 415 response
        // you can inject the authenticator to mock authentication results while ensuring you test true to how
        // the call will be made
        final ExampleAuthenticator authenticator = new ExampleAuthenticator();
        addProvider(new ExampleSecurityProvider<>(authenticator));
    }

    @Test
    public void getAll() throws Exception {
        List<User> users = client().resource("/user")
                                   .header(ExampleSecurityProvider.CUSTOM_HEADER, "validAdminToken")
                                   .get(new GenericType<List<User>>() {});
        assertEquals(2, users.size());
        assertEquals("user1", users.get(0).getUsername());
    }

    @Test
    public void getAllThrows401WhenNotAuthenticatedToken() throws Exception {
        try {
            client().resource("/user")
                    .get(new GenericType<List<User>>() {});

            fail("Should have thrown 401");
        } catch (UniformInterfaceException ex) {
            assertEquals(ex.getResponse().getStatus(), 401);
        }
    }
    ...

Conclusion

Using the injectable security provider you can implement an extremely flexible authentication and authorisation system in your service, while keeping the code overhead in your resource methods and tests minimal via the Auth attributes. In terms of simplicity it blows away a lot of security libraries I’ve worked with for other frameworks.

API performance testing using different implementations

Done a simple performance test of APIs using different implementations. Could be useful for example scripts and for anyone looking to do something similar.

Lovely how easy and cheap EC2 makes it to do proper testing against proper scaled environments.

https://github.com/stevenalexander/simple-api-performance-testing