Debian-news is about one simple thing - news about Debian GNU/Linux and the top free distributions based on Debian GNU/Linux.


Bits from an FTP Master: New team member, need more volunteers, (kind of) todo list

Hello everyone, I thought it was time again to send a mail to d-d-a, and looking for an excuse, I found the ftpteam.

The first good excuse is the addition of a new member to our team.
Everybody please send your condolences to Luca Falavigna, dktrkranz.
He joined us a while ago as a trainee and is now an FTP Assistant.

And while I have your attention, let me tell you that we need your help!

I’m starting with a call for volunteers but will follow it with a (kind
of) todo list which interested people can work on. And, while some of the
jobs can only be done by team members, many can be done without joining
the team, and a few can even be worked on by people who aren’t Debian
Developers (yet).

If you are interested feel free to contact us at!

Of course, as with any team, a few points you should think about (not
all apply for all tasks, of course):

– You need to be able to deal with all the existing team members.
(And they with you). :)

– You need to be able to deal with sometimes unpopular decisions. People
do not always like it when you reject a package, but that is no reason
to allow bad ones in. If you can’t stand a bit of flames / don’t like
to take hard decisions, this is no job for you. (But look out for the
other possibilities :) ).

– You must love to read and deal with legal texts, like licenses and
their effect on the package in Debian. The ftpteam is *the* one place
to decide if something is ok for Debian to distribute or not, and you
will have to take this decision.

– You need a very good understanding of the archive, how packaging works,
know qa processes and the general way things are dealt with in
Debian. This job will throw you right in the middle of all this.
You have to know the basics of just about every programming language
you can imagine (and all those you can’t but that are still there),
of all the different packaging systems people use. NEW will present
all of them and more stuff you never heard about and you need to be
able to dig through it, searching for possible bad things.

– You should be able to read and write python. At least if you want to
understand the code behind our archive, or even want to help
maintaining it.


So of course the one task that is seen by most people, and as such is
mentioned first, is NEW processing. (You have to be a DD for this!)
The current team of assistants is doing a very good job keeping the
process running; packages usually pass NEW in a very short time. Of
course there are always exceptions, not all packages are simple, but
overall they just rock!

Debian is not getting smaller and the NEW queue is a kind of Sisyphean
task – uploads that want to get processed never stop. As such, more
manpower to spread the load is very much wanted, the more, the better
(to a limit, but we aren’t at that limit yet).

What will happen when you decide to volunteer for NEW (and rm) processing?

Simple: You get added to our ftptrainee group. This got setup some while
ago and is a simple but effective way to let you find out if this is
work you actually want to commit to and at the same time enables us to
see you working and help you learn the rules we have in NEW. The way
this setup works is simply letting trainees access the ftpmaster machine
and the NEW queue. You can look at packages and their source as any other
team member. But trainees can not do the actual ACCEPT or REJECT. Instead
you have a special ability to leave notes about the packages, explaining
what action you would take and why. The other team members will then
review those notes and either follow your advice or tell you why they
decided to do something different.

After a while we and you will know if you actually fit the team, but
more important we (and you yourself) will know if you should (want to)
continue doing NEW and will promote you up to assistant. We set ourself
a time limit of 6 months as a maximum stay in the trainee group, but
none of the current team members has ever stayed in trainee that long.
The longest is 3 months, the shortest is 6 days.


Another good way helping the ftpteam is – by not joining us! Yvan eht
Nioj! (Or for people not watching Simpsons: Join the Navy!)
Err, no, of course not, but: Join the QA team!

There is a lot of work commonly associated to the QA group but not many
people doing it. You could help out there, which in turn will help us


Do various coding work. This is the one point where even non-Debian
Developers can help (in many cases). Some cases of course need Developer
status at least, but not all of them.

We have a lot of open todo points, presenting them all here would probably
only get this mail rejected due to its size, so I will randomly select
a few. If you happen to have something of your own that really really
needs to get into dak, fine, speak up, im open. :)

First: Our git is at (http and https, both work)

and the list associated with it is

and our IRC channel for development is #debian-dak on

As our code is mainly python with a few shell and perl additions, you
should be familiar with python. We are using sqlalchemy for database
access, so knowledge there does not hurt. And if coding needs changes
to the database schema, SQL is very helpful, so you should have basic
knowledge there as well.

– Improve our testsuite.
This is a task that can be done even if you are not a Debian Developer
yet. Our code has a good start at a test suite already since our
ftpmaster meeting in Essen, but this can surely be extended a lot.

– General code cleanup.
Every codebase that is as old as dak now is loves people cleaning up.
Have fun. :)

– Contents files, Packages files
For quite a while we have been working on changing the way we create
and update our Contents files. And when this is done the generation of
Packages/Sources files should follow suit. Basically, this will move
the extraction of the data for Contents (and then Packages/Sources)
files from the few dinstall runs we have per day over to the many many
upload processing runs we have per day. And dinstall will only write
out the pre generated data from our database.

Due to some issues not related to the ftpteam, the person who was
mainly working on this task is not available to do it, so someone
should take over. Much of the work for the contents part is done, it
mainly needs testing and final integration. The Packages/Sources part
that follows will work on top of this later on.

– Autosigning
There are plans to allow buildds to automatically sign packages. There
already have been discussions between the buildd folks, DSA and
ftpmaster how this can work out (DSAed buildds, certain firewall and
access restrictions, special keyrings, special upload queues), but the
details aren’t very interesting for this mail. Just – we need a little
work on the archive side to make it work. Much is, again, already
prepared, the patchset developed during our Essen meeting allows
certain restrictions we have to use. But there are still a few things
to be thought over and written, and here you (as a DD, sorry, needs host access I think) can help.

– Throw away DD uploaded .debs.
The initial dependency of this task, having lintian based automated
rejects, is in use for some time now, so we can do the final steps for
this task now. This needs a bit of work, from the top of my head:
– Needs a way to define a build-architecture for arch: all debs. Some
of them can only build on certain architectures. Maybe a control file
– Needs a way to define exceptions. Both, based on suites (we do not
want to throw away security uploads to stable!) and maybe also per
package or per gpg key.

– debianqueued seriously needs a replacement.
This is the perl monster that every uploader knows. It is the tool
that forwards uploads to localhost, after you put them up with ftp. It
is really really old perl code that got modified many times. It is
near to unmaintainable and really should get a replacement.

There have been many ideas, including It doesn’t
need to be this, of course. Feel free to discuss with us any ideas
you have, but something smart would be nice. For example if we could
move various checks we do to an upload to here we could reject various
packages even before they are uploaded. (Easy examples are: expired
key in the keyring we use or the upload is for an older version,
the archive knows a newer one).

There are millions more on the todo list, but for now I will stop.

— bye, Joerg .SH AUTHOR This manual page was not written by anyone. It sprang forth into existence on its own.

One Response to “Bits from an FTP Master: New team member, need more volunteers, (kind of) todo list” »

  1. Pingback by Bits from an FTP Master: New team member, need more volunteers … | Just linux! — March 13, 2010 @ 1:40 am

    […] Read more:  Bits from an FTP Master: New team member, need more volunteers … […]

RSS feed for comments on this post. TrackBack URI

Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Debian-News is not related to the Debian Project.
All logos and trademarks on this site are property of their respective owners.