HTTP Status Report

A nice lecture by Mark Nottingham about HTTP.
Highly recommended!

Verifying Signatures of the Query Parameters

Recently I have spent few hours trying to understand why the verification of the digital signature of the parameter, which was sent as query parameter on the url, fails.

I put two logging messages on the parameter: one at request as it arrives, before any parsing is done at all and one just before the signature verification.

When verifying the first output, the verification was successful, but the second one was failing. "What a voodoo!" - thought I, and decided to compare this string in text edit. You know to take one string, type "Search" and try to find the second one. It succeeded. "Voodoo!" - thought I again. How the strings can be identical in a text editors and have a different signature?

If you want to think about it, just stop reading here and think.

Personally I started thinking about some hidden characters and so on... But the answer was much simple: Text editor compare text ignoring the case. It's so obvious that we usually don't think about it. Some (very few) of the characters of the text had a different case and that's why the text editor found them, but the signature failed.

However, another question was left: How happened that the query parameter, which was sent on the url, changed the case of some of its characters? The answer is a bit tricky. The parameter that is sent on the url should be url-encoded. The sender was encoding the url before creating a digital signature. The receiver was aware of this functionality and moreover he was also aware that the servlet container decodes the parameters as it receives them. So the receiver was encoding the parameter back to url-encoded format before validating the signature and everything worked.
Until one day came another sender, who did exactly the same logic, but who used a different library for the url encoding. He used lower case characters for the encoded values instead of the upper case characters that are usually used by most of the libraries.

So the scenario for the incorrect behavior is clear. And where is the defect?
The defect is in the receiver's part of code. Receiver should never get parameters from the servlet container, but get the whole query string before it's going to be parsed and do the parsing by himself. It's incorrect to do the parsing and then to correct the parsing errors by url-encoding the parameter back to its original state. The parameter should be verified as is.

How to cause the IE to open specific application based on mime type

In general IE opens files based on their extensions. So when you try to access let's say http://localhost/myapp/helloWord.csv, it will open excel.

This is good, but now good enough, since the content on the web usually is not defined by file extensions, but by a content-type header. Especially in REST, when a resource can have different representations and we want same URL for resource:
http://localhost/myapp/helloWord should return HTML, http://localhost/myapp/helloWord?alt=application/atom+xml should return atom, http://localhost/myapp/helloWord?alt=text/csv should return csv and so on.
So what will happen if we ask for CSV in this way and server returned content-type=text/csv? The IE will not know how to handle such response and it will ask the user. However, we expect it to understand that this is CSV file and it should be opened by Excel. Right?

So there are two solutions for this problem:

1. Teach IE that mime text/csv should be mapped to the .csv file. This can be done in registry:

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Mime\Database\Content Type\text/csv]
"Extension"=".csv"

By the way in this way you can map and mime to any extensions. Most of the mime already predefined, but some (like CSV) are missing.

2. Send Content-Disposition header.
If the response contains Content-Disposition: attachment; filename=HelloWorld.csv, it will cause the IE to open Excel. It happens since this header actually means that the response should be saved as HelloWorld.csv file.

Let's compare these two solutions:
First solution is better, when you cannot change the response. The response arrives, it doesn't contain Content-Disposition header, but it contains the proper mime type. So the mapping is done at client and this is fine.

The second solution is better, when you can change the server. It's obviously better to add additional header instead of changing registry of each client.

Passing Query Parameters over HTTP

When you start to learn HTML Forms you learn something like:

method - The HTTP method for sending data to the action URL. Default is get.
method="get": This method sends the form contents in the URL: URL?name=value&name=value.

method="post": This method sends the form contents in the body of the request.
(taken from http://www.w3schools.com)

Pretty clear, right?

But what happens when you use POST and add parameters to URL:

form action="URL?name=value&name=value" method="POST"

Will it work?
The answer is yes. It will. Some of the parameters will be passed on URL and some in the body and it will be transparent for the server side.

Is it OK? It looks like bug...
Well, it isn't bug. The HTTP spec (RFC 2616) defines HTTP url as

http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]]

for all HTTP verbs. Including POST.
It's HTML spec that defines how the method POST should treat a form data: to put it in the message body and so on; and there is no problem at all to add another query params to the URL.

And what happens if you want to POST a real body? Something that isn't parameters, may be atom message to update Blogger or something?
How would the server know that the body of the message is a body and not a list of parameters?

So here are the rules:

SRV.4.1.1 When Parameters Are Available
The following are the conditions that must be met before post form data will
be populated to the parameter set:
1. The request is an HTTP or HTTPS request.
2. The HTTP method is POST
3. The content type is application/x-www-form-urlencoded
4. The servlet has made an initial call of any of the getParameter family of methods
on the request object.
If the conditions are not met and the post form data is not included in the
parameter set, the post data must still be available to the servlet via the request
object’s input stream. If the conditions are met, post form data will no longer be
available for reading directly from the request object’s input stream.
(From Servlet Specification 2.3)

In addition to these rules, the parameters are not available in case of chunking. I didn’t find this part in spec, but I tested it and it just doesn’t work.

So it as it appears, it's very difficult to get a POSTed body as parameter. You must not use chunking, you must define content type as application/x-www-form-urlencoded and you must to call any of the getParameter family of methods on the request object. Right? It looks very difficult, but it what the most of the people do by default. It was exactly what I did, when I invoked a very simple HTTP call from java application.

So for the clients the best solution, in my opinion, is always ensure that you set proper content type. That's the way you tell the server: the body cotains parameters or the body contains the real body.

For servers the solution is: always call getParameter BEFORE starting reading the body. Checking content type can be also helpful.

Very nice visualization of HTTP status codes:

http://thoughtpad.net/alan-dean/http-headers-status.jpg

via http://thoughtpad.net/alan-dean/http-headers-status.html
via some guy, who mailed me these nice links.