Thursday, December 8, 2011

Integration Testing of RESTful Application

In the previous posts I have described how to start Jetty from code in the beginning of the unit tests and how to initialize the in-memory HSQL database once Jetty is started.

If you have completed the steps from these posts, you should have a running application in the beginning of your tests. And now you are ready to start the actual testing of the application.

I believe that the best way to test RESTful API is to issue actual HTTP requests and since we have a Jetty server running, it becomes possible.

There are a lot of HTTP Clients available in Java. The examples below use Apache HTTP Client.
But as always first let's add a maven dependency:

<dependency>
         <groupId>org.apache.httpcomponents</groupId>
         <artifactId>httpclient</artifactId>
         <scope>test</scope>
        </dependency>

Now some convenient static methods that can be used:

import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpDelete;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.methods.HttpPut;
import org.apache.http.entity.ByteArrayEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.AbstractHttpMessage;
import org.apache.http.message.BasicHeader;
...
   static HttpResponse executeGet(String url) throws IOException {
        HttpClient httpclient = new DefaultHttpClient();
        HttpGet get = new HttpGet(url);
        return httpclient.execute(get);
    }

    static HttpResponse executeDelete(String url) throws IOException {
        HttpClient httpclient = new DefaultHttpClient();
        HttpDelete delete = new HttpDelete(url);
        return httpclient.execute(delete);
    }

    static HttpResponse executePost(String url, byte[] body) throws IOException {
        HttpClient httpclient = new DefaultHttpClient();
        HttpPost post = new HttpPost(url);
        post.setEntity(new ByteArrayEntity(body));
        return httpclient.execute(post);
    }

    static HttpResponse executePut(String url, byte[] body) throws IOException {
        HttpClient httpclient = new DefaultHttpClient();
        HttpPut put = new HttpPut(url);
        put.setEntity(new ByteArrayEntity(body));
        return httpclient.execute(put);
    }

So your test can be something like:

@Test
    public void testGet() throws Exception {
        HttpResponse response = executeGet(url, null, null, null);
        assertEquals(response.getStatusLine().getStatusCode(), 200);
    }


Recommended Reading

1. Next Generation Java Testing: TestNG and Advanced Concepts
2. Apache Maven 3 Cookbook
3. Spring Recipes: A Problem-Solution Approach

Automated Integration Tests Using with Jetty, Maven and Other Neat Freameworks - 2

In the previous post, I have started a Jetty server in the beginning of a unit test with a configured external data-source.

Let's talk about it a little bit more. The assumption here was that the application on a regular basis uses an external data-source that is accessed via JNDI. In general it's a good practice to keep a data-source external to the application:
1. It's always possible to change the data-source without touching the application - let's say a bug was found in a data-source you are using. Or it should be configured differently. If a data-source embedded into an application and such a change is required, you will probably need to release patch. If the datasource is external, it would be enough just to change/reconfigure it.
2. In some deployments several applications can use the same datasource. Consider a Tomcat running with several wars: quite a common case, right? If a datasoure is embedded, each war has its own data-source. Usually this would be a waste of resources and duplication of a configuration.
3. And finally it would be possible to change a data-source for unit tests, which is exactly our use case.

For unit tests it's really convenient to use in-memory database. There are few databases implemented in java that provides this capability. My favorite are H2 and HSQL. In this example I have used HSQL, so here comes the jetty-ds-test.xml:

<Configure id="Server" class="org.eclipse.jetty.server.Server">
    <New id="DSTest" class="org.eclipse.jetty.plus.jndi.Resource">
        <Arg></Arg>
        <Arg>jdbc/my_ds</Arg>
        <Arg>
            <New class="org.hsqldb.jdbc.JDBCDataSource">
                <Set name="Url">jdbc:hsqldb:mem:test;sql.syntax_ora=true</Set>
                <Set name="User">sa</Set>
                <Set name="Password">sa</Set>
            </New>
        </Arg>
    </New>
</Configure>

This is configuration of HSQL in memory. Notice sql.syntax_ora=true, which makes HSQL to use Oracle syntax. This is useful if you are using Oracle in production, but you want to use HSQL for unit testing or development.

So now you have at the beginning of a unit-test you have a Jetty server running with your application connected to in-memory HSQL database via JNDI. But something is still missing. This something is a database schema: to remind you, we have just started a new database in memory and it's empty. You probably already have a script that generates schema, tables, indexes and may be some data. Now you need to run it.

Actually there are several options how do it. One of the easiest is to use Springs's SimpleJdbcTestUtils. But first we'll need to add a dependency on spring-test in our pom.xml:

<dependency>
         <groupId>org.springframework</groupId>
         <artifactId>spring-test</artifactId>
         <scope>test</scope>
         <version>${spring-version}</version>
</dependency>

And here comes the code that runs the sql script:
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.sql.DataSource;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;
import org.springframework.jdbc.core.simple.SimpleJdbcTemplate;
import org.springframework.test.jdbc.SimpleJdbcTestUtils;
...
    @BeforeClass(dependsOnMethods = { "startJetty" })
    public void initiateDatabase() throws Exception {       
        InitialContext initialContext = new InitialContext();
        DataSource ds = (DataSource) initialContext.lookup("jdbc/my_ds");
        SimpleJdbcTemplate simpleJdbcTemplate = new SimpleJdbcTemplate(ds);
        Resource resource = new ClassPathResource(sqlScriptFileName);
        SimpleJdbcTestUtils.executeSqlScript(simpleJdbcTemplate, resource, false);
    }
In this snippet I load the sqlScriptFileName from the classpath. Usually it's convenient to place the script in src/test/resources, but if you don't like it, you can always load it from a different place by using other Resource implementations (e.g. URLResource is quite convenient).

As I have already said in this snippet I used Spring. If you are familiar with Spring, it is probably natural to you. If you don't - don't be afraid. Only the unit tests become dependent on Spring, but the actual application did not.

And now you are ready to start the testing.


Recommended Reading

1. Next Generation Java Testing: TestNG and Advanced Concepts
2. Apache Maven 3 Cookbook
3. Spring Recipes: A Problem-Solution Approach

Automated Integration Tests Using with Jetty, Maven and Other Neat Freameworks

Let's say that you have a web application (aka war) that exposes RESTful API and connects to a database. Now you want to do some automation tests (and I'm not going to describe here why actually you must have automation tests that run regularly on your API).

One of the best solutions to do it, in my opinion, is running your application on a Jetty server that is embedded in your unit test with some in-memory database.

But let's do it step-by-step:

Step1: Start Jetty from a Unit Test

(In this guide I have used TestNG, but I see no reason why the same functionality cannot be achieved in JUnit)

So first we need to start Jetty in our test. But before we actually do that, we need to make sure that our Maven project contains the relevant dependencies. So first we need Jetty:
<dependency>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-server</artifactId>
            <version>${jetty-version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-webapp</artifactId>
            <version>${jetty-version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-jndi</artifactId>
            <version>${jetty-version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-plus</artifactId>
            <version>${jetty-version}</version>
            <scope>test</scope>
        </dependency>

The Jetty dependencies in this guide already contains the JNDI support. It will be needed later. But if JNDI support is not required, they can be omitted.

And then let's start it before the tests start:

import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.webapp.WebAppContext;
import org.eclipse.jetty.xml.XmlConfiguration;
import org.testng.annotations.BeforeClass;

public class MyTest {

    private static final String RESOURCES_URL = "/rs";
    private static final String CONTEXT       = "/app_context";
    private static final String DS_CONFIG     = "/jetty-ds-test.xml";
    private String              baseResourceUrl;

    @BeforeClass
    public void startJetty() throws Exception {
        Server server = new Server(0);   // see notice 1
        server.setHandler(new WebAppContext("src/main/webapp", CONTEXT)); // see notice 2

        // see notice 3
        InputStream jettyConfFile = InboxTest.class.getResourceAsStream(DS_CONFIG);
        if (jettyConfFile == null) {
            throw new FileNotFoundException(DS_CONFIG);
        }
        XmlConfiguration config = new XmlConfiguration(jettyConfFile);
        config.configure(server);

        server.start();
        
        // see notice 1
        int actualPort = server.getConnectors()[0].getLocalPort();
        baseResourceUrl = "http://localhost:" + actualPort + CONTEXT + RESOURCES_URL;
    }
Please notice that:
1. Jetty is started on a random port. The actual url with the actual port is saved to baseResourceUrl to be used later by tests.
2. Web application context points to maven's src/main/webapp.
3. Jetty is started with a data source configuration. (See Runnig Jetty from Maven using JNDI Data Source)

Part 2


Recommended Reading

1. Next Generation Java Testing: TestNG and Advanced Concepts
2. Apache Maven 3 Cookbook
3. Spring Recipes: A Problem-Solution Approach

Monday, December 5, 2011

Runnig Jetty from Maven with JNDI Data Source

During the development of a java web component (aka war) it can be very useful to run the application as quick as possible. Jetty provides a Maven plugin that allows running it directly from maven build or explicitly using "mvn jetty:run" from the command line.

But what happens if the war uses external database? Especially when the datasource is defined externally to the war and accessed vie the JNDI?

Actually the solution if quite simple:

Step 1 - Define the Data-source

Create the file defining the data-source:
(The example below is for Oracle. See this page for examples of the other databases.
<Configure id="Server" class="org.eclipse.jetty.server.Server">
    <New id="DSTest" class="org.eclipse.jetty.plus.jndi.Resource">
        <Arg></Arg>
        <Arg>jdbc/my_ds</Arg>
        <Arg>
            <New class="oracle.jdbc.pool.OracleDataSource">
                <Set name="DriverType">thin</Set>
                <Set name="URL">jdbc:oracle:thin:@(DESCRIPTION=(ENABLE=BROKEN)(ADDRESS=(PROTOCOL=TCP)(HOST=127.0.0.1)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=something_to_replace)))
                </Set>
                <Set name="User">my_user</Set>
                <Set name="Password">my_password</Set>
                <Set name="connectionCachingEnabled">true</Set>
                <Set name="connectionCacheProperties">
                    <New class="java.util.Properties">
                        <Call name="setProperty">
                            <Arg>MinLimit</Arg>
                            <Arg>10</Arg>
                        </Call>
                        <Call name="setProperty">
                            <Arg>InactivityTimeout</Arg>
                            <Arg>600</Arg>
                        </Call>
                    </New>
                </Set>
            </New>
        </Arg>
    </New>
</Configure>

Pay attention to "jdbc/my_ds". It's the datasource JNDI name. Make sure to put there the actual JNDI name used by your application.

Place this file under your maven project. In my opinion the best is to place this file under src/dev/resources, but basically it can be anywhere.

To continue the example, I'll name the file: jetty-ds-dev.xml

Step 2 - Define the Jetty Maven Plugin

<build>
...
        <plugins>
...
            <plugin>
                <groupId>org.mortbay.jetty</groupId>
                <artifactId>jetty-maven-plugin</artifactId>
                <version>${jetty-version}</version>
                <configuration>
                    <jettyConfig>src/dev/resources/jetty-ds-dev.xml</jettyConfig>
                </configuration>
                <dependencies>
                    <dependency>
                        <groupId>com.oracle</groupId>
                        <artifactId>ojdbc14</artifactId>
                        <version>${oracle-ojdbc-version}</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>

Pay attention to the configuration of jetty-ds-dev.xml.
Also pay attention that Jetty must be able to find the relevant JDBC driver in its classpath!

And basically that's all. Run "mvn jetty:run" and your application should work with the provided database.


Recommended Reading

1. Apache Maven 3 Cookbook
2. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation