While reading the HTTP/1.1 and HTTP/2 specs, I had to admit that I didn’t know anything about socket programming and everytime I used a snippet of code I felt like a chicken that just found a knife.
But this morning I stumbled upon this really nice overview in the Python documentation, and I wanted to share it with you because it helped me a LOT !
Nevertheless I am even more curious now, so what resources would you recommend to learn socket programming ?
Have a nice weekend !
Beej’s Guide to Network Programming is kind of the classic socket programming read.
When did that get written? I usually think of Stevens as being the classic. His TCP/IP illustrated is also seminal.
Beej strikes me more as a highly condensed crash course while Stevens gets more in-depth on everything. There’s a place for both.
I guess classic for people in my mid-30s cohort. Beej’s went around the university hacking club 15 years ago and I still look at it when I have to dive back in to C sockets..
It’s gotta be a generational thing. When I was getting started (’92), Stevens was de rigueur.
Read it, re-read it, and internalize it.
And it’s fully accessible online, wow ! I will definitely read this one, thank you :)
/thread
At the same time you learn about socket programming, learn to use wireshark – wireshark is a network protocol analyzer, it will capture traffic and let you see what messages are going back and forth (including protocol headers). Use netcat and/or your preferred language’s socket library to do various things on the network, and see what the traffic looks like. That should complement reading about the protocols, it’ll help to verify whether your socket library is doing what you want, and wireshark is a great tool for investigating all kinds of network issues.
I learned from beej’s network guide (C-based), followed by Steven’s Advanced Programming in the Unix Environment and The Linux Programming Interface. There’s a lot of overlap between those two (the latter has more coverage of Linux-specific extensions), but you may find one or the other clicks with you better.
I remember using Wireshark during my studies… Thanks for the idea, I will use it !
I can really recommend UNIX Network Programming, Volume 1: The Sockets Networking API (3rd Edition)
It’s from 2003 but still relevant today. It’s very concise and complete in my opinion and more to the point than Advanced Programming in the Unix Environment when it comes to socket programming.
It’s older than that, isn’t it? I have a copy on my shelf and I’m almost positive I bought it in 1998. I still go to it on an irregular basis… it’s no stretch to call it my longest-lived useful tech book right next to Cormen/Leiserson/Rivest’s algorithms text.
Amazon lists it at over 1000 pages.
It is indeed a huge book, but has concise coverage of a zillion different Unix APIs.
Those statements are not mutually exclusive.
I did this stuff in my first professional programming job (working on the DPOP mail server, mostly, in C) and it’s a deep and broad field.
Do you mean sockets, or TCP/IP? Two different things. If you’re learning protocols like HTTP, then what you need to know is the latter: the way Internet networking works. Addresses, ports, packets, connections. (And you don’t need to know it to much depth; the algorithms in the TCP implementation are interesting, but you almost never need to know how they work to get stuff done.)
The sockets API is the most common API for using IP, but it’s not the only one, and it’s used for non-IP stuff too (like Unix IPC.) It is IMHO a pretty nasty API at the C level, with all sorts of gotchas and incompatibilities. The abstracted version of it in Python is probably a lot cleaner since it doesn’t have to deal with low level issues like signals, and can build on a robust stream API.
It’s confusing that the word “socket” is used for both this specific API and for a TCP connection.
Thanks for the clarification ! (As you expected, I focus on TCP/IP for now)
I wrote a book on the topic. It’s called Hands-On Network Programming with C. It starts with the basics, and it uses lots of examples to illustrate concepts. It’s in C though, not Python.
HTTP/2 is going to be really complicated to learn because there’s a whole lot of tangent algorithms you’ll need, such as HPACK. I’d recommend starting with HTTP/1.x. The spec is fairly easy to follow. I’ve written some posts over the years on writing an HTTP/1.x server from the socket layer. Here’s one in Node.js. You can find similar posts online for various languages.
Yeah, HTTP/2 is a very different beast. In January, I started to undestand the HTTP/2 spec by porting Hyperframe from python to Nim (Hydra) But in the end, I had so many gaps in networking that I paused it.
I will definitely read your blog posts, thanks !
I would get a book on TCP/IP to understand the protocols (I originally learned from TCP Illustrated vol. 1, 1st edition by Stevens; some people seem less happy with 2nd edition though, which was updated by a different person).
Then, for the OS-level APIs, Linux Programming Interface (https://nostarch.com/tlpi) is an amazing reference on everything Linux (or really Unix—it covers both Linux specific and POSIX and other standard features, so it’s useful for any Unix-y system). It has a ~150 pages on socket APIs, select(), epoll(), etc.. If you’re doing system programming in Unix and you like paper books, you should own this book.
There are great suggestions in this thread. I would second Beej’s guide as an intro, but then read the man pages. Wireshark is a good tool to use, if you are going to be at the command line I would suggest getting familiar with tcpdump (the -XX option to display hex dumps is great) and use netcat.
I was recently deploying gunicorn as a service and found Beej’s Guide (as mentioned above) when I discovered that I had no idea what a unix socket was. It was a fun way to find an exceptional guide.
Thank you very much everyone, I wasn’t expecting that much advices ! I definitely a lot of things to read now :)