Friday, September 16, 2011

Asynchronous non blocking curl multi example in C++

The curl documentation makes me want to cry.

The library seems nice enough and if you want to do some blocking synchronous http requests (and print them out to the standard out) then it's extremely straight forward.

But who wants to do blocking requests? There are very few programs that can get away with locking entirely until a http request is done.

Also one my internet pet-peeves, when searching for solutions I often come upon threads or mailing lists with someone stating my exact problem and then a reply saying read the docs. It would be so much nicer, internet, if the question had been answered right there and then.

Anyway here's the rough equivalent of simple.c in the examples but using the multi interface so it doesn't block.

The header.
class Http
 CURLM* multi_handle; 
 int handle_count;

  void Update();
  void AddRequest(const char* uri);

The implementation.
 multi_handle = curl_multi_init();
 handle_count = 0;


void Http::Update()
 curl_multi_perform(multi_handle, &handle_count);

void Http::AddRequest(const char* uri)
  CURL* curl = NULL;
  curl = curl_easy_init();
  curl_easy_setopt(curl, CURLOPT_URL, uri);
  curl_multi_add_handle(multi_handle, curl);

First thing to note - the curl pointer is never released! That needs to be taken care of.

The code can be used like

Http http;

// In some update loop called each frame
That will then print the google homepage out to the standard out. Also a good guide to building Curllib can be found here:


Anonymous said...


Its seems a nice post however I have one dumb question how I can play with the data returned ? Do I still need to put a do While loop in some thread ? like

int still_running=0;

do {
while(::curl_multi_perform(multi_handle, &still_running) ==
} while (still_running);

Dan Relluchs said...

Yeh, something like that.

Most programs I write are games, so all games have a loop like you describe. Which makes polling solution great.

If you're making more event-driven program then there may be better ways to do it using CURL(I can't really remember the specifics - it's been a while since I wrote this post)

Anonymous said...

Thanks Dan for instant reply I think I have understood the asynchronous means of curl_multi_init :).

What actually I got is that I can have a pThread over my while loop or I can do it in my GameSchedule. And I can use CURLOPT_WRITEDATA option to read all the data.

So all the files I ask multi handle to download will internally be async and my pThread will be making the whole downloader as Non Blocking.

Kindly hit me if I said anything wrong :).