1 May 2012 20:15
Re: Setting priority for an easy handle in a multihandle
Gokhan Sengun <gokhansengun <at> gmail.com>
2012-05-01 18:15:03 GMT
2012-05-01 18:15:03 GMT
How does this "real time TCP/IP stack" let you do that in a way that a "normal" stack doesn't?We need to get on the same page. In order to get a hard real time effect, you need operating system support, it has to provide you a real time TCP/IP stack. What I suggested can only work on the "best effort basis". It can only help you to avoid procesing a low priority jobs while high priority job is waiting for attention.
Above I am not arguing a "real time TCP/IP stack" alone can help a better performance of course. Instead I am emphasizing what can be done with the TCP/IP stack on hand. I just wanted to make sure that there is no hard/firm real time requirement here. If there is, one needs a preemptive kernel, hard real-time aware device driver, real-time TCP/IP stack, etc. More integrated solution (both HW/FW/SW) then a standard setup.
Then you need to make your second select() exclude the low-priority sockets as otherwise they will just trigger select() immediately again - and this is of course not easily done with libcurl within a single multi handle and that's a reason for you to use multiple ones.That is the whole point here. After returning from select, my code first checks whether high priority job needs action, if so action is provided and without doing any more work we are selecting again with the minimum expiry time.
No I am not excluding low-priority sockets and they trigger select immediately. Then if no prior job is ready for processing, I am processing just one low-priority job and select again. For the time being, my tests show that "latency" of processing high priority jobs in an acceptable range but in the future, if bosses do not like it, then we will sit and add features to libcurl so that it does everything atomically for multi interface. For example, send command "USER xxxx" to FTP server, break FTP state machine and return back to multi interface before waiting the response, etc.Currently processing of a job in multi interface for FTP does USER, PASSWD, PWD and returns and this can take some time to complete if poles of earth are involved in the communication. So this is not today's discussion for me and my application does not have hard or firm real time requirement.
If bosses will be interested in a quick but a little dirty solution, I am going to be select'ing for like 10 ms with prior descriptors to favor them if code feels a possible activity in prior descriptors.
With the multi_socket() API it would possibly be easier to do within a single handle.
I also inspected multi_socket() API when I started my implementation but we are living in an embedded world and life is not that easy here. I have to handle everything (new requests, network processing of existing requests) inside a single thread and I have many more interesting requirements that I can hardly do with multi_socket() and to be honest I did not want to mess with an event library and add another complexity into my code.
Hm, come to think about it. Perhaps your way can also be achieved with some clever use of curl_easy_pause() on the low-prio transfers while there are high-prio ones to deal with...
I was not aware of this feature, yes it looks it can be useful.
Aah yes, that's how you can "sleep too long" since you then enforced a sleep onto the low prio transfers that they didn't ask for.This goes on until select wakes up with no higher priority job needs immediate attention, then we process low priority job.
As said, I do not enforce sleep for the time being.
------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
RSS Feed