curl POST data and newlines

I’m writing a Rails application for the microblogging strategy for Debian (which I used to call, debian-twitter, but you know how us Debian folks are :D). I came up with a strange issue, that at first I thought was ActionController’s fault or some crap.

So, I was testing one of my controllers with something like this:

cat /tmp/mm.txt | curl -d @- http://localhost:3000/message/new

/tmp/mm.txt is a PGP signed message which then I just send as POST data to my application. So far so good. However, when accessing the data from Rails, using request.body.read, I was a getting a single line, with the newlines (carriage returns) removed. So I started looking at how ActionController was getting the HTTP data and stuff. But then I tested:

cat /tmp/mm.txt | lwp-request -m POST  http://localhost:3000/message/new

And that had the carriage returns in place.

So, I started looking at the curl man page and discovered this little gem:

–data-binary

(HTTP) This posts data exactly as specified with no extra processing whatsoever.

If you start the data with the letter @, the rest should be a filename. Data is posted in a similar manner as –data-ascii does, except that newlines are preserved and conversions are never done.

If this option is used several times, the ones following the first will append data as described in -d/–data.

cat /tmp/mm.txt | curl --data-binary @- http://localhost:3000/message/new

…did indeed the tricky. Maybe someone needs this info some day.