The Call for Speakers (and registration) for the Linux Plumbers Conference is open! Get those proposals in, register, and come join us in sunny Portland, OR.
The Linux Plumbers Conference was created to bring together the key developers involved in Linux plumbing - the “Linux plumbers” - and give them an opportunity to discuss problems face-to-face, both within subsystems and across subsystems. Participants include invited attendees, speakers selected through an open, competitive review process, and students. Registration is open to the general public as well.
…
The goal of the Plumbers Conference is to solve problems. The conference is arranged as a series of microconferences, each on a topic that is narrow enough to identify specific problem areas and brainstorm workable solutions. Each microconference is led by an expert in the field and organized to encourage discussion and problem solving.
I’ve been watching metalink for a while now, and was urged to write about it, so I am! If you want to download something large that isn’t available via bittorrent, you can still get the distributed download benefits of bittorrent. Basically, metalink-aware downloaders split up your download across many mirrors of the resource you want, using existing protocols.
Bittorrent is great for “new releases” or other similiar things that are currently “in demand” by a large number of people. For older stuff, especially large trees of files, as found on many mirrored archives, you won’t find a bittorrent, and metalink can really distribute and speed up the download.
OpenOffice is using it, and I hope to figure out a way to incorporate it into kernel.org directly. There are already places hosting auto-generated metalink files for various projects, including the linux kernel. I’m hoping kernel.org can publish more complete metalink files since we should be able to build them more easily, having the list of which mirrors are in which countries, their access mechanism, and if they carry bz2, gz, or both. We’ve talked about it briefly, but haven’t finalized the plans yet.
You can even generate your own metalink files online.
Another blog has other details, so I hope I’ve not been too redundant. :)
The “sendfile” system call is a way to send file contents directly out to a network socket. This saves time in userspace (so it doesn’t have to copy buffer contents around), and was one of the reasons I upgraded kernel.org’s Apache to version 2.x at the end of 2003 (because version 1.x doesn’t have sendfile support). A few weeks ago, one of the other kernel.org admins discovered that files greater than 2G were not being delivered by Apache.
I had a lot of fun tracking down the issue. The “amount to send” argument in the sendfile call is a “size_t”, which is basically an “unsigned long”. Having a 2G limit didn’t make sense, since even with 32 bits, that should be a 4G limit. However, the kernel.org servers are both 64bit, so as it turns out, “size_t” is a full 64 bits. After writing a quick test, I was able to verify that it was, indeed, a 31 bit limit on both 64 bit and 32 bit kernels. Peter Anvin took it from here, and tracked down the origin of the problem: filesystem operations greater than 31 bits in offset were being rejected deep in the kernel. He suggested truncating the request instead of returning a failure.
Seems as though Linus decided to limit the size of filesystem calls to make sure there aren’t security problems (signed vs unsigned overflows) in the various filesystem drivers, while people using the Linux kernel migrate more from 32bit to 64bit systems. Personally, I don’t agree with this, but from a practical stand-point, it hardly makes a difference. Instead of sending all 4G out the pipe and returning to user space, it just returns twice, sending 2G per call.
This should be fixed in 2.6.16. Until then, we could patch Apache to keep it’s offset request under 31 bits, but we’ll probably just tell people to use FTP, since vsftpd doesn’t use sendfile yet.
I joined the OSUOSL guys to meet master.kernel.org as it was flown in from California this morning. The master server’s new home is just above a stack of Mozilla servers at the OSL’s facilities. Great place. Nice people (some with blogs). Lots of bandwidth. :)
Afterwards, I got invited to OSU’s annual “Geeks and Steaks” picnic. That was great, and I got to meet all sorts of other folks that work in the IT groups at OSU and stand around watching them light the BBQ. No liquid oxygen, but, then, they all wanted to keep their arm hair, I guess.
As if a blog itself wasn’t enough shameless self-promotion, I’ve just gotta share more! There’s a nice write up at Kernel Trap on how the kernel.org servers work. I’m mentioned in there since I’m one of the guys that helps keep the machine running. Mostly I manage the mirror server validations. I’m the guy that bugs the other mirror admins about falling behind, etc. The real work is done by HPA and Nathan. They’ve got some great gear in place now. I can’t wait until the next Fedora Core release; it really drives the machines hard. I think it’s great to log into a fairly responsive machine, and see a load average over 200. And then realize that that’s normal. I love it!
