Need to design a web crawler using C language, the specifications for this assignment is in that project 1 web crawler pdf. There’s two additional documents for starting tips.
My tutor said the sort of http libraries that you shouldn’t use are libcurl. (as an exception, you can use the HTML parser htmltidy.c from libcurl.)
Also they said client.c could be a good starting point, most of them actually use that file to modify to make it work as a crawler.