20:05:09 <MadameZou> #startmeeting 20:05:09 <MeetBot> Meeting started Thu Nov 18 20:05:09 2010 UTC. The chair is MadameZou. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:05:09 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic. 20:05:31 <Hatrix> #help 20:05:33 <liw> Hi, let's start. 20:05:38 <liw> Please have your tuna fish sandwiches ready. 20:05:44 <liw> This is an interactive tutorial for making Debian packages. 20:05:49 <liw> I show and tell things, and you ask questions. 20:05:55 <liw> Questions are welcome, at any time. 20:06:04 <liw> However, to keep this channel focused, let's use a different channel for questions. 20:06:07 <liw> #dw-question is the channel. 20:06:17 <liw> No talk in this channel during the tutorial, please. 20:06:22 <liw> MadameZou will paste questions from that channel back to this one, one-by-one. 20:06:27 <liw> I'll answer each question before the next one gets pasted. 20:06:34 <liw> To simplify this, if you have a question, start it with "QUESTION:". 20:07:11 <liw> This will be an introductory tutorial. 20:07:19 <liw> I will not get very deep into the more intricate bits of Debian packaging. 20:07:27 <liw> I will show how to make Debian packages for software that is simple to package. 20:07:38 <liw> I will use only the dh command from debhelper 7. 20:07:54 <liw> Over the 17 years of its existence, Debian has come up with a lot of tools for packaging. Some of them layered deep. 20:08:09 <liw> My favorite by far is dh, from debhelper version 7 onwards. 20:08:16 <liw> So that's what we'll cover today. 20:08:28 <liw> I will assume that you understand installation of binary packages. 20:08:36 <liw> Also, general command line use, and editing of text files using whatever text editor they like: emacs, vi, gedit, kate, anything is fine. 20:08:46 <liw> You should have the packages build-essential, devscripts, and debhelper installed. 20:08:51 <liw> The versions in lenny should be good enough, but I'm using squeeze. 20:09:17 <liw> that was a lot of text, fairly fast... is anyone having trouble keeping up? I'll try to go slower, so there's time to read and think 20:10:09 <liw> Let's start with some central concepts. 20:10:32 <liw> The three central concepts here are upstream tarball, source package, and binary package. 20:10:37 <liw> Most people package software that someone else has written. 20:10:49 <liw> This someone else is called the upstream developer. 20:10:57 <liw> They will make a release of their software, and the concrete result of that is the "upstream source archive file", or tarball. 20:11:07 <liw> A tarball is the ".tar.gz" or ".tgz" file upstream makes. 20:11:15 <liw> ("tarball" is a bit of computer jargon that you will run into a lot when making Debian packages.) 20:11:26 <liw> The tarball is what Debian takes and builds a package out of. 20:11:34 <liw> Sometimes there is no upstream tarball, but we'll not get into that now. 20:11:39 <liw> Sometimes you are the upstream developer, but we'll not get into that now, either. 20:11:45 <liw> Let's keep things simple, and learn the typical simple case first. 20:12:02 <liw> So you have the upstream tarball. 20:12:06 <liw> The next step is to make the Debian source package. 20:12:13 <liw> From this, we can then build the Debian binary package, which is what actually gets installed. 20:12:24 <liw> The source package consists of the upstream tarball, plus some files that you yourself create. 20:12:32 <liw> The source package consists, in its simplest form, of three files. 20:12:37 <liw> 1) The upstream tarball, renamed to follow a systematic pattern. 20:12:47 <liw> 2) A patch file, with any changes made to to upstream source, plus all the files created for the Debian package. This has a .diff.gz ending. 20:13:06 <liw> 3) A description file (with .dsc ending), which lists the other two files. 20:13:11 <liw> This sounds a bit more complicated than is necessary, at first sight. 20:13:16 <liw> It would be simpler to just have everything in one file. 20:13:39 <liw> However, it turns out that it saves a lot of disk space and bandwidth to keep the upstream tarball separate from any Debian specific changes. 20:13:48 <liw> It's also easier to keep track of what has been necessary to change for Debian. 20:14:02 <liw> This in turn makes it easier to report any bug fixes and other improvements made in Debian to the upstream developer. 20:14:09 <liw> The packaging workflow is usually like this: 20:14:16 <liw> 1) rename the upstream tarball to follow a specific naming pattern 20:14:23 <liw> 2) unpack the upstream tarball 20:14:29 <liw> 3) add the Debian packaging files (in a subdirectory called debian), and make any other necessary changes 20:14:35 <liw> 4) build the source package 20:14:41 <liw> 5) build binary package 20:14:50 <liw> 6) upload the source and binary packages to Debian 20:14:59 <liw> Some people say you should do some testing before the final step. 20:15:09 <liw> Testing is what those people do who can't everything right the first time. 20:15:18 <liw> Er, I guess we all need to do that, then. 20:15:32 <liw> any questions so far? all clear? 20:16:14 <MadameZou> QUESTION: quilt? 20:16:27 <liw> quilt is a tool for managing patches 20:16:50 <liw> I'll mention it at the end, again, but it's outside of the narrow scope for this tutorial; but we can discuss it more at the end 20:17:11 <MadameZou> another question liw :) 20:17:14 <liw> yep 20:17:23 <dapal> QUESTION: you are talking about the official way to do it. Is it ths same when I do my own little inofficial stuff? 20:18:10 <liw> it is pretty much the same for one's own little inofficial packages, though one does not need to care about quite every detail; I'll point out some in the examples soon 20:18:23 <liw> next question 20:18:38 <MadameZou> QUESTION: Can I generate a multi-arch package? 20:18:48 <heeed> evenin 20:19:16 <liw> I don't think Debian supports multi-arch packages, yet, but I'll cover "Architecture: all" packages later 20:19:20 <liw> next question 20:19:25 <dapal> QUESTION: please clarify about packaging those packages that are already in binary form, i.e. nvidia blob or likewise 20:20:04 <liw> for binary blob packages, you treat the tarball with the blogs as the source package, and just avoid compiling them from source; I've never had to do this, though 20:20:28 <liw> that seems to have been the last question for now, so I'll go forward 20:20:31 <jane_mt> QUESTION: is there anything like ppa in ubuntu? 20:20:48 <liw> questions to #dw-question please, but I'll answer this one now 20:21:27 <liw> Debian does not run a PPA service like Ubuntu, but all the tools to make apt repositories yourself are there, it's just a lot to configure oneself 20:21:34 <liw> In order to not be completely theoretical, let's walk through an actual example. 20:21:40 <liw> There is a sample application at http://liw.fi/hithere/ 20:21:44 <jordan___> QUESTION: the log of the channel will be available later? 20:22:03 <liw> Please download the upstream source tarball, you'll have to follow a link to find it. 20:22:15 <liw> This is good practice for packaging things for real: some upstreams hide the upstream tarball very, very well. 20:22:18 <dapal> jordan___: yes it will be, questions to #dw-question please 20:22:25 <liw> So, step 0 is to download the tarball. 20:22:31 <liw> # wget http://code.liw.fi/hithere/hithere-1.0.tar.gz 20:22:43 <liw> Step 1 is to rename it. 20:22:50 <liw> The Debian packaging tools assume the upstream tarball has a very specific name. 20:22:57 <liw> The name consists of the source package name, an underscore, the upstream version number, followed by ".orig.tar.gz". 20:23:01 <liw> The source package name is should be all in lower case, and can contain letters, digits, and dashes. 20:23:12 <liw> Some other characters are also allowed, but let's stick to those for now. 20:23:17 <liw> If the upstream developer uses a name that looks like a good Debian source package name, then you should use that. 20:23:24 <liw> Otherwise, change the upstream name as little as possible to make it fit for Debian. 20:23:29 <liw> In this case, upstream has wisely picked a suitable name, "hithere", so we don't need to worry about it. 20:23:37 <liw> What a smart upstream. 20:23:44 <liw> We should end up with a file called hithere_1.0.orig.tar.gz 20:23:50 <liw> Note that there is an underscore, not a dash, in the name. 20:23:55 <liw> This is important, because the packaging tools are picky. 20:24:00 <liw> # mv hithere-1.0.tar.gz hithere_1.0.orig.tar.gz 20:24:14 <liw> Step 2 is to unpack the tarball. 20:24:19 <liw> We should have a directory called "hithere-1.0". 20:24:26 <liw> In general, the sources should go into a directory named after the source package name and upstream version, with a hyphen (not underscore) in between. 20:24:33 <liw> This is again because the packaging tools are picky. 20:24:57 <liw> (There's a reason they are picky. Things work better that way in the long run.) 20:25:03 <liw> In this case, upstream has again been wise and made the tarball pack into the right subdirectory. 20:25:13 <liw> # tar xf hithere_1.0.orig.tar.gz 20:25:19 <liw> Step 3 is to add some Debian packaging files. 20:25:26 <liw> All of these files go into the debian/ subdirectory inside the source tree. 20:25:39 <liw> # cd hithere-1.0 20:25:44 <liw> # mkdir debian 20:25:48 <liw> There are a bunch of files we need to provide. 20:25:52 <liw> I'll walk through them in order. 20:25:58 <liw> First file is debian/changelog. 20:26:02 <liw> This is the log of changes to the Debian package. 20:26:08 <liw> It does not need to list everything that has changed in upstream code, though it can be helpful to users to summarize those changes, too. 20:26:19 <liw> Or not, if that is a lot of text. 20:26:22 <liw> Since we are now making the first version, there is nothing to log. 20:26:28 <liw> However, we still need to make a changelog entry, because the packaging tools read certain information from the changelog. 20:26:34 <liw> Most importantly, they read the version of the package. 20:26:39 <liw> debian/changelog has a very particular format. 20:26:45 <liw> (picky, picky, picky) 20:26:54 <liw> The easiest way to create it is to use the dch tool. 20:26:57 <liw> # dch --create -v 1.0-1 --package hithere 20:27:01 <liw> This will result in a file like this: http://paste.debian.net/100134/ 20:27:30 <liw> A couple of notes. 20:27:33 <liw> The "hithere" part MUST be the same as the source package name. 20:27:40 <liw> "1.0-1" is the version. The "1.0" part is the upstream version. 20:27:47 <liw> The "-1" part is the *Debian* version: the first version of the Debian package of upstream version 1.0. 20:27:55 <liw> If the Debian package has a bug, and it gets fixed, but the upstream version remains the same, then the next version of the package will be called 1.0-2. 20:28:03 <liw> Then 1.0-3, and so on. 20:28:09 <liw> "UNRELEASED" is called the upload target. 20:28:20 <liw> It tells the upload tool where the binary package should be uploaded. 20:28:25 <liw> "UNRELEASED" means the package is not yet ready to be uploaded. 20:28:41 <liw> It's a good idea to keep the "UNRELEASED" there so you don't upload by mistake. 20:29:02 <liw> (Been there, done that. Got a good deal on brown paper bags.) 20:29:07 <liw> Ignore "urgency=low" for now. Just keep it there. 20:29:13 <liw> The "(Closes: #XXXXXX)" bit is for closing bugs when the package is uploaded. 20:29:18 <liw> This is the usual way in which bugs are closed in Debian: when the package that fixes the bug is uploaded, the bug tracker notices this and marks the bug as closed. 20:29:24 <liw> However, we don't need to worry about that now. 20:29:33 <liw> We can just remove the "(Closes...)" bit. Or not. It doesn't matter right now. 20:29:36 <WillyKaze> Hi there 20:29:37 <liw> The final line in the changelog tells who made this version of the package, and when. 20:29:54 <liw> The dch tool tries to guess the name and e-mail address, but you can configure it with the right details. 20:29:59 <liw> See the dch(1) manual page for details. 20:30:04 <liw> That's it for the changelog. 20:30:09 <liw> The next file is called debian/compat. 20:30:16 <liw> This file should contain the number 7. 20:30:21 <liw> This is a magic number. 20:30:26 <liw> Do not put any other number in there. 20:30:32 <liw> Otherwise your package may transmogrify into an MS-DOS installation floppy. 20:30:37 <liw> Or, more realistically, it will probably not build. 20:30:44 <liw> (debian/compat specifies the "compatibility level" for the debhelper tool. We don't need to go into what that means, right now.) 20:31:54 <liw> question please 20:32:08 <MadameZou> < captnfab> QUESTION: do we need to repack the original tarball if it 20:32:09 <MadameZou> doesn't contains a properly named foo-1.0 folder? 20:32:12 <MadameZou> ops, sorry 20:33:13 <liw> I think that these days, you don't need to repack it. It used to be necessary, many many years ago, but now the "dpkg-source -x" command will name the directory the right way, if necessary. 20:33:16 <liw> next question 20:33:17 <MadameZou> QUESTION: do we need to repack the original tarball if it doesn't contains a properly named foo-1.0 folder? 20:33:32 <MadameZou> yeah, it was the previous :/ 20:33:44 <liw> no worries 20:33:55 <MadameZou> QUESTION: can you explain better why things works better changing underscore in hypen in the orig.tag.gz file? 20:34:52 <liw> the dpkg-dev tools make assumptions on the name of the various files; this makes them simpler to implement, and simplicity makes things work better in general 20:35:02 <liw> next question, please 20:35:13 <MadameZou> < dnf> QUESTION: how can i install dch tool? 20:35:30 <MadameZou> erm 20:35:36 <liw> dch is in the devscripts package; you will need build-essential and debhelper as well 20:35:42 <liw> next question, please 20:37:03 <MadameZou> liw: just a moment, there's a very synthetic formulation :) 20:37:48 <MadameZou> QUESTION: I can see "official" DEB packages in 20:37:49 <MadameZou> /var/cache/apt/archives without changelog and compat files. Why? 20:38:24 <liw> the .deb packages in /var/cache/apt/archives are binary packages; the changelog is included in them (under a different name), the compat file is only in the source package 20:38:41 <liw> next question, please 20:39:32 <MadameZou> iiw said the the compat file should comtain the magic number 7, I assume this just means as single char files as produced by 'echo 7 >deabian/compat' ? 20:39:50 <liw> yep, a single character 7 is enough to have in debian/compat 20:40:14 <liw> next question, please 20:40:21 <MadameZou> QUESTION: I've heard of non-maintainer specific format 20:40:22 <MadameZou> for debian revision numbers 20:41:21 <MadameZou> ehm, the question is all here ^ 20:41:42 <liw> I think you mean either the "native package" format (in which the version number would be just 1.0, not 1.0-1); or the "non-maintainer upload" format, in which case that's too much detail for this session to get it right in all cases, but "1.0-1.1" would be an example 20:41:47 <liw> next question, please 20:42:24 <MadameZou> QUESTION: When is "dpkg-source -x" run? Is it done manually? 20:43:03 <liw> "dpkg-source -x" is the command to unpack a Debian source package after it has already been built (usually by someone else); it is not usually run when creating the package 20:43:10 <liw> sorry, should have clarified that earlier 20:43:24 <liw> ok, going forward 20:43:29 <liw> You'll notice during this tutorial there seems to be a lot of magic going on. 20:43:34 <liw> Do not read http://en.wikipedia.org/wiki/Clarke%27s_three_laws (it's a distraction). 20:43:39 <liw> Also do not read http://marcel-oehler.marcellosendos.ch/comics/ch/1986/06/19860614.gif (it's another distraction). 20:43:46 <liw> I'm skipping a lot of details to concentrate on the things that are important to know right now. 20:43:52 <liw> You can get back to the magic bits later. 20:43:59 <liw> The magic is not particularly advanced technology, it is just details we do not need to understand know. 20:44:04 <liw> It's a good idea to come back to the magic bits later, to understand what is really going on, but for now, we need to get something working at all first. 20:44:24 <liw> debian/control is a bit less magic. 20:44:27 <liw> http://paste.debian.net/100138/is an example of what it might look like. 20:44:33 <liw> The control file describes the source and binary package, and gives some information about them, such as their names, who the package maintainer is, and so on. 20:44:41 <liw> There are several required fields, but you can just treat them as magic, for now. 20:44:47 <liw> So, in debian/control, there are two paragraphs. 20:44:51 <liw> The first one describes the source package. 20:44:54 <liw> The "Source:" field gives the source package name. 20:45:01 <liw> "Maintainer:" gives the name and e-mail address of the person responsible for the package. 20:45:11 <liw> "Build-Depends:" lists the packages that need to be installed to build the package. 20:45:18 <liw> They might or might not be needed to actually use the package. 20:45:29 <liw> The second paragraph describes the binary package built from this source. 20:45:34 <liw> Actually, there can be many binary packages built from the same source. 20:45:37 <liw> But we will only build one now. 20:45:52 <liw> "Package:" is the binary package name. 20:45:57 <liw> The name might be different from the source package name. 20:46:02 <liw> "Architecture:" tells which computer architectures the binary package is expected to work on: i386 for 32-bit Intel CPUs, amd64 for 64-bit, armel for ARM processors, and so on. 20:46:09 <liw> Debian works on about a dozen computer architectures in total, so this architecture support is crucial. 20:46:23 <liw> The "Architecture:" field can contain names of particular architectures, but usually it contains one of two special values. 20:46:30 <liw> "any" (which we see in the example) means that the package can be built for any architecture. 20:46:36 <liw> In other words, the code has been written portably, so it does not make too many assumptions about the hardware. 20:46:43 <liw> However, the binary package will still need to be built for each architecture separately. 20:46:49 <liw> The other special value is "all", which means that the same binary package will work on all architectures, without having to be built separately for each. 20:46:57 <liw> For example, a package consisting only of shell scripts would be "all". 20:47:06 <liw> Shell scripts work the same everywhere and not need to be compiled. 20:47:37 <liw> question, please 20:47:43 <KatharsisFreak> Question: This control file will not work with the debhelper version in lenny? 20:48:36 <liw> KatharsisFreak, please ask in #dw-question 20:48:48 <MadameZou> liw: we have some questions ;) 20:48:56 <liw> MadameZou, yep, fire away 20:49:33 <MadameZou> QUESTION: how can one determine the needed packages to 20:49:35 <MadameZou> be used by Build-Depends 20:50:04 <liw> I usually do that by a) trial and error and b) reading the source of the program 20:50:15 <liw> both are usually necessary 20:50:22 <spion> .oO(or documentaion) 20:50:23 <liw> aste next question, please 20:50:26 <dapal> QUESTION: can i build package for armel from my i386, and can i do it easy? 20:50:57 <liw> that would be cross-compilation, and I don't think there is an easy to do it; there are tools for it, but I am not familiar with them, sorry 20:51:00 <liw> paste next question, please 20:51:49 <dapal> <vibora> QUESTION: and what's the difference between Depends and Build Depends, in what case i have to use only Depends instead of Build-Depends 20:52:21 <liw> Build-Depends are needed _only_ during package build, while it is being compiled, and its test suite is being run 20:52:37 <liw> Depends is needed _only_ when the package is actually being used, after being installed 20:53:02 <liw> some things might be both build and runtime dependencies, and in that case you need to put them into both Build-Depends and Depends 20:53:49 <vibora> ok, that covers perfectly my next question ;) 20:54:02 <liw> as an example, if a package contains a PHP script, but it does not get run while the package is built, then PHP would go into Depends only, not Build-Depends 20:54:18 <sc> QUESTION: may I have a Build-Depends that does not have a corresponding Depends? I'm thinking to statical linked libraries 20:54:21 <liw> however, if the PHP script is only used to run a test, and does not get inclued in the binary package, it would only be in Build-Depends and not in Depends 20:54:51 <liw> sc, please ask in #dw-question, but I'll answer this one right now 20:55:04 <sc> liw sorry 20:55:43 <liw> Build-Depends and Depends are pretty much entirely separate, and it is perfectly ok (from a packaging tool point of view) to have dependencies only in one and not the other, or in both 20:55:50 <liw> paste next question please 20:55:54 <dapal> <KatharsisFreak> Question: This control file will not work with the debhelper version in lenny? 20:56:20 <liw> I did not test these files with lenny, sorry; I was under the impression they should work there, but they might not 20:56:25 <liw> paste next question please 20:56:51 <dapal> <pauli> QUESTION: when i build a package with "Architecture: all" field, some of the resulting file (b.build, .changes) still have the build architecture in the name. Is a right behaviour? 20:57:06 <liw> yep, that is the right behavior, no need to worry about it 20:57:11 <liw> paste next question please 20:57:12 <dapal> <fernandomerces> QUESTION: Is there another package for source package or its the same of binary package? I will generate and mantain two packages (binary and source)? 20:57:54 <liw> you edit the source package and then run a command (which I will get to in a bit) to build the binary package out of the source package 20:58:11 <liw> I hope that part gets clearer when we get to the end of the example 20:58:12 <liw> paste next question please 20:58:17 <dapal> <fernandomerces> QUESTION: It's not clear to me if "Architecture" options will make effect over source package (building process) or binary package (install and run). 20:59:08 <liw> the "Architecture:" field tells whoever is building the package whether there is any point in building the package on a particular computer architecture 20:59:19 <liw> so if it says i386 and amd64 only, then there's no point in building it on amd64 21:00:00 <liw> the, er, entity paying most attention to it are the automatic build servers Debian uses, which look at Architecture to see if they need and can build a package 21:00:11 <liw> Debian has at least one build server for each architecture, often several 21:00:15 <liw> paste next question please 21:00:22 <dapal> liw: no more questions 21:00:32 <liw> ah, good, going forward then :) 21:00:44 <Tekk_away> oh cool, I'm not TOO late? 21:00:51 <liw> The "Depends:" field lists the packages that must be installed for the program in the binary package to work. 21:01:01 <liw> Listing such dependencies manually is tedious, error-prone work. 21:01:02 <dapal> liw: there's the one about ${shlibs:Depends}, but I suppose you're going to explain that right now :-) 21:01:08 <liw> It is therefore a perfect job for computers. 21:01:14 <liw> To make this work, the "${shlibs:Depends}" magic bit needs to be in there. 21:01:30 <liw> The other magic bit is there for debhelper. The "{misc:Depends}" bit. 21:01:39 <liw> sorry, more magic stuff 21:02:04 <liw> the shlibs magic is for shared library dependencies, the misc magic is for some stuff debhelper does 21:02:33 <liw> for other dependencies, you need to add them manually to Depends or Build-Depends 21:02:48 <liw> oh, and the ${...} magic bits only work in Depends 21:02:56 <liw> (that's a simplification, but good enough for now) 21:03:01 <liw> "Description:" is the package description. 21:03:04 <liw> It is meant to be helpful to users. 21:03:08 <liw> Even though the example entirely fails to do so, it is a good idea to spend some effort into making it actually helpful to the user, so they will know what the package will do for them, and can make an informed decision of whether to install it or not. 21:03:22 <liw> That's it for debian/control. Save the example and let's continue. 21:03:25 <liw> debian/copyright is the next file. 21:03:40 <liw> It is quite an important file, but for now we will be happy enough with an empty file. 21:03:44 <liw> For Debian, this file is used to keep track of the legal, copyright-related information about a package. 21:03:53 <liw> However, it is not important from a technical point of view. 21:03:57 <liw> For now, we'll concentrate on the technical aspects. 21:04:00 <liw> We can get back to debian/copyright later, if there's interest. 21:04:05 <liw> debian/rules is also a magic file. 21:04:08 <liw> It should look like this: http://paste.debian.net/100139/ 21:04:22 <liw> The last line should be indended by one TAB character, not by spaces. 21:04:27 <liw> The file is a makefile, and TAB is what the make command wants, for reasons that will require a time machine to really figure out. 21:04:32 <liw> (If you have any handy cardboard boxes, please use one and return give us the answer to the question.) 21:04:42 <liw> debian/rules can actually be quite a complicated file. 21:04:45 <liw> However, the dh command in debhelper version 7 has made it possible to keep it this simple in many cases. 21:04:57 <liw> This is the primary reason why I like dh. 21:05:05 <liw> We'll see if it works this time. I don't know if it does. 21:05:09 <liw> Let's hope I can stand the suspense. 21:05:12 <liw> (http://marcel-oehler.marcellosendos.ch/comics/ch/1991/04/19910406.gif) 21:05:20 <liw> The final file we need is debian/source/format, and it should contain the version number "1.0". 21:05:25 <liw> This is the version number for the format of the source package, and even though it happens to be the same as the upstream version, that is just a co-incidence. 21:05:34 <liw> Just treat it as an other bit of magic. 21:05:57 <liw> All right, that should be all we need. 21:06:01 <liw> Now we can build the package. 21:06:07 <liw> The command to do that is "debuild -us -uc". 21:06:10 <liw> There are many commands we could use for this, but this is the one we'll use. 21:06:16 <liw> If you run the command, you'll get output similar to this: http://paste.debian.net/100144/ 21:06:23 <liw> Something went wrong. 21:06:46 <liw> There is actually a lot more output than is on pastebin, but I've extracted the important part at the end of the total output. 21:06:54 <dapal> liw: question for you 21:06:55 <liw> I'll get to that, I see we have a question. 21:07:04 <dapal> <vorlon> and <vibora> QUESTION: isn't source format 3.0 the magic to be used these days? 21:08:21 <liw> yes, the 3.0 format is the new magic, but the 1.0 version works quite as well for the examples, and for people's initial packages, and does not require me to consider whether to teach "3.0 (native)", "3.0 (quilt)", "3.0 (git)" or something else :) 21:08:38 <liw> getting back to the example 21:08:44 <liw> There is actually a lot more output than is on pastebin, but I've extracted the important part at the end of the total output. 21:08:48 <liw> This is what usually happens. 21:08:56 <liw> You do your best creating debian/* files, but there's always something that you don't get right. 21:09:03 <liw> Not to worry, the intrepid Spaceman Spiff will come to the rescue! 21:09:08 <liw> Er, sorry about that. Didn't mean to get quite so excited. 21:09:13 <liw> I'm feeling better now. 21:09:16 <liw> So, the thing that went wrong is this bit: 21:09:19 * dapal hands some tea to liw 21:09:22 <liw> install hithere /home/liw/debian-packaging-tutorial/x/hithere-1.0/debian/hithere/usr/local/bin 21:09:32 <liw> The upstream Makefile is trying to install the compiled program into the wrong location. 21:09:38 <liw> Actually, that looks like a very weird location. 21:09:43 <liw> There's a couple of things going on here. 21:09:48 <liw> First is a bit about how Debian packaging works. 21:09:54 <liw> When when program has been built, and is "installed", it does not get installed into /usr or /usr/local, as usual, but somewhere under the debian/ subdirectory. 21:10:01 <PaulRufous> what package in format 3.0? 21:10:11 <liw> We create a subset of the whole filesystem under debian/hithere, and then we put that into the binary package. 21:10:18 <liw> So the .../debian/hithere/usr/local/bin bit is fine, except that it should not be installing it under usr/local, but directly under usr. 21:10:22 <liw> We need to do something to make it install into the right location (debian/hithere/usr/bin). 21:10:30 <liw> The right way to fix this is to change debian/rules so that it tells the Makefile where to install things. 21:10:35 <liw> http://paste.debian.net/100145/ is the preferred way to do that. 21:10:46 <liw> It's again a bit of magic, and to understand it you'll need to know how Makefiles work, and the various stages of a debhelper run. 21:10:53 <liw> For now, I'll summarize by saying that there's a command debhelper runs that takes care of installing the upstream files, and this stage is called dh_auto_install. 21:11:23 <liw> We need to override this stage, and we do that with a rule in debian/rules called override_dh_auto_install. 21:12:08 <liw> The final line in the new debian/rules is a bit of 1970s technology to invoke the upstream Makefile from debian/rules with the right arguments. 21:12:16 <liw> If hithere had been written by someone who actually knows how to do things right, there would have been a properly written configure script, and then everything would have worked automatically, without the override. 21:12:22 <liw> But I guess from a pedagogical point of view it was good to learn about the overrides, even if they look a bit scary. 21:12:29 <liw> ok, questions now, please 21:12:43 <dapal> <lilith> QUESTION: that example debian/rules contains only #!make -f , what to do with ./configure , or cmake or scons or .. anything else part? 21:13:22 <liw> dh has a lot of heuristic, so a package with a ./configure script, or one of the other common ways of building packages, will usually build without any additions 21:13:42 <liw> my example program does not have one of the common ways, so it needs a bit of tweaking 21:14:00 <yang_> clear 21:14:09 <liw> continuing 21:14:16 <liw> Let's try to build things again. 21:14:21 <liw> Oh no! It still fails! 21:14:26 <liw> This time, this is the failing command: 21:14:30 <liw> install hithere /home/liw/debian-packaging-tutorial/x/hithere-1.0/debian/hithere/usr/bin 21:14:36 <liw> We are now trying to install into the right place, but it does not exist. 21:14:41 <liw> See for yourselves: ls debian/hithere 21:14:45 <liw> To fix this, we need to tell the packaging tools to create the directory first. 21:14:50 <liw> Ideally, the upstream Makefile would create the directory itself, but in this case the upstream developer was too lazy to do so. 21:15:16 <liw> The packaging tools (specifically, debhelper) provide a way to do that. 21:15:51 <liw> I was just told that lenny has a version of debhelper that is older than what the Build-Depeds I show has, but if you replace the version number with 7.0 it might work 21:16:24 <liw> Create a file called debian/hithere.dirs, and make it look like this: http://paste.debian.net/100149/ 21:16:31 <liw> The second line is creates the directory for the manual page. We will need it later, I'm shortcutting a bit. 21:16:38 <liw> Now the build succeeds, but there's still some small problems. 21:16:48 <liw> debuild runs the lintian tool, which checks the package that has been built for some common errors. 21:16:54 <liw> It reports several for this new package: http://paste.debian.net/100150/ 21:17:00 <liw> These should eventually be fixed, but none of them look like they'll be a problem for trying the package. 21:17:07 <liw> So let's ignore them for now. 21:17:11 <liw> Look in the parent directory to find the package that was built. 21:17:15 <liw> # ls .. 21:17:18 <liw> Output: http://paste.debian.net/100152/ 21:17:40 <liw> we'll discuss those things in a moment, but first, we have a question 21:17:44 <PaulRufous> please, slower 21:17:52 <liw> PaulRufous, ack, I'll go slower 21:19:34 <dapal> <lilith> QUESTION: heuristics is good, but what if we need to pass --with-something-special to configure or cmake arguments, does heuristics handle those source packages that need ./waf and other tools to build ? 21:20:41 <liw> I like dapal's answer: 21:20:42 <liw> <dapal> lilith: using dh7, if you want to pass something to ./configure, just do something like: 21:20:42 <liw> <dapal> override_dh_auto_configure: 21:20:42 <liw> <dapal> <TAB>dh_auto_configure -- --your-params 21:21:13 <liw> dh has a nice extension/override mechanism that allows you to override particular parts; it is pretty easy to use, once you've read the docs 21:21:28 <liw> ok, continuin with the example 21:21:38 <liw> The following command will install the package that you've just built. 21:21:43 <liw> Do NOT run it on a computer you don't mind breaking. 21:21:54 <liw> In general, it is best to do package development on a computer that is well backed up, and that you don't mind re-installing if everything goes really badly wrong. 21:22:05 <liw> Virtual machines are a good place to do development. 21:22:10 <liw> That said, here's the command: 21:22:15 <liw> # sudo dpkg -i ../hithere_1.0-1_amd64.deb 21:22:22 <liw> Output: http://paste.debian.net/100151/ 21:22:25 <liw> So, that seemed to work, at least for me. 21:22:46 <dapal> <spion> QUESTION: where can we find docs for override stuff of debhelper 7? 21:22:53 <dapal> (good question, eh. :-D) 21:23:01 <liw> yes, it is 21:23:57 <liw> dh runs a specific sequence of commands to build a package; each command is called dh_something, and each such command can be overridden by having a debian/rules entry called override_dh_something 21:24:14 <liw> add the --no-act option to the dh command in debian/rules to see what commands it runs 21:24:22 <spion> ah, cool 21:24:45 <liw> you then have to read the manual page for that command to see if you can configure it (via a file such as debian/hithere.dirs), or if you need to override it 21:24:50 <liw> paste next question please 21:24:51 <dapal> <bognu> QUESTION: from where do you usually run your commands? inside the dir hithere-1.0 or outside? should there be an outer dir called hithere (without number) 21:25:24 <liw> I always run the packaging commands in the hithere-1.0 directory, and i usually have a directory above that called hithere 21:25:43 <liw> but the parent directory is not necessary, it is just neater, so files from different projects don't get mixed up 21:25:54 <liw> paste next question please 21:26:09 <dapal> reluctant to paste this, but... 21:26:10 <dapal> <yang_> QUESTION: i must not have done something right because my BIN dir is empty in package data.tar.gz and i have no binary for hithere anywhere 21:27:12 <liw> yeah, something must gone wrong, yang; let's postpone debugging that, but I'll mention that the #debian-mentors channel is supposed to be a place to ask such questions, and a number of people hang out there willing to help you learn to package and help you debug these kinds of problems 21:27:21 <liw> going forward 21:27:31 <liw> How do we test the package? 21:27:36 <liw> Why, we can run the command. 21:27:41 <liw> # hithere 21:27:46 <liw> It works! 21:27:51 <liw> Well, it works for me. 21:28:09 <liw> Even the manual page works. 21:28:17 <liw> I declare this package a success. For me. 21:28:36 <liw> But, it's not perfect. 21:28:41 <liw> Remember, lintian had things to say, and debian/copyright is empty, etc. 21:28:51 <liw> We have a package that works, but it isn't yet of the high quality that is expected of a Debian package. 21:28:59 <liw> There's probably a whole lot of other things that can be fine-tuned. 21:29:04 <liw> However, let's not forget what we do have a working Debian package. 21:29:09 <liw> Adding 7 fairly simple files to the upstream source is all it takes. 21:29:12 <liw> Or at least that's what it takes for simple packages. 21:29:20 <liw> There are programs that are quite hard to make packages for. 21:29:26 <liw> Most programs, however, are quite easy. 21:29:31 <liw> There's a reason for that: Debian developers like to package all the software they use, so the packaging process is pretty simple, after you get over the initial learning curve. 21:30:09 <liw> However, packaging something like a web application, or any very large piece of software, is going to be quite challenging, even for experienced deveopers. 21:30:14 <liw> That is not a reason to not try. 21:30:20 <liw> It's a reason to allocate more time for it than just one evening. 21:30:25 <liw> And it is always OK to ask for help if you get stuck. 21:31:01 <dapal> <fernandomerces> QUESTION: I have a upstream that needs a qmake (uses Qt) before make. Can I add it to debian/rules? How? 21:31:03 <liw> That ends my lecture. 21:31:37 <tixy> Thank you liw 21:31:41 <liw> I think you need a override_dh_auto_build rule in debian/rules, but I haven't built any Qt packages, so I am not sure 21:31:46 <vibora> liw: thanks a lot for this great tutorial!! 21:31:46 <spion> liw: COOL, MANY THANKS 21:31:47 <liw> The floor will soon open for questions and discussion. 21:31:50 <liw> But before that, I'll have a couple more things to point out. 21:31:51 <spion> hups 21:31:56 <TetsuyO> thank you very much liw 21:31:57 <liw> Once you've built your own packages, you'll eventually want to learn how to set up your apt repository, so your package is easy to install. 21:32:02 <liw> The best tool for that I know of is reprepro. 21:32:10 <liw> For more testing of your package, you may want to look at the tool called piuparts. (I wrote it originally so it is perfect and never has any bugs. er...) 21:32:16 <liw> And, finally, if you start making changes to the upstream source, you'll want to learn about the quilt tool. 21:32:25 <TetsuyO> and also thanks to the moderators :) 21:32:28 <liw> And, finally, if you start making changes to the upstream source, you'll want to learn about the quilt tool. 21:32:33 <liw> All of these things are big enough that they won't fit into this tutorial, but it's good to know about. 21:32:38 <liw> Other things you might want to read are listed on the http://www.debian.org/devel/ web page. 21:32:43 <liw> http://code.liw.fi/hithere/ also has a tarball with the contents of the debian/ directory in my example. 21:32:47 <liw> OK, discussion and questions now. 21:32:52 <MadameZou> < bognu> QUESTION: I have a project, where I make the 21:32:52 <MadameZou> deb package "by hand" more or less. I dont have a lot of 21:32:52 <MadameZou> things you mentioned... but lintian tells me no 21:32:52 <MadameZou> warnings. So what das this mean for me? Is the package really fine then? 21:33:42 <liw> bognu, it means your package is probably fine; like I said at the beginning, Debian's packaging tools have a 17 year old history, and there are many tools, and many ways in which to do things; every way that works is fine 21:33:47 <liw> next question, please 21:33:49 <MadameZou> < valhalla> QUESTION: what is a good workflow to update 21:33:51 <MadameZou> a package to a new upstream version? 21:35:10 <liw> ah, I am not sure what is the best workflow for that, but bascially: unpack the upstream source to a new directory, copy the debian/* directory from the old package, update debian/copyright ("dch -v 1.2.3-4") and try to build, and fix anything that is broken 21:35:30 * ibid likes uupdate for that 21:35:33 <spion> s/copyright/changelog/ 21:35:36 <liw> that's not a really good workflow, but for that you will need to learn quilt, and possibly how to use version management with debian packaghing, and that's too big a topic for this session, sorry 21:35:44 <liw> spion, er, yes, you're right 21:35:54 <liw> there are too many files beginning with c in there 21:35:59 <spion> :) 21:36:03 <liw> next question, please 21:36:08 <MadameZou> < sdd> QUESTION: Could you estimate the amount of work 21:36:08 <MadameZou> to get to being an average packager? (how long did it 21:36:08 <MadameZou> take you to a state where you could release, resp. teach) 21:37:14 <liw> I learned this stuff some time last decade, so I have a very bad idea of how easy it is to learn, never mind using modern tools 21:37:48 <liw> but I suspect if you have access to people you can ask, and your first packages are fairly simple, then a few evenings should be enough to make a working package you don't mind sharing with others 21:37:53 <liw> next question, please 21:37:55 <MadameZou> < acr> QUESTION: source packages may be builded with 21:37:56 <MadameZou> some options to configure script, what official way to 21:37:56 <MadameZou> determine from binary package what options used? 21:38:20 <liw> the official way is to find the corresponding source package and see what it uses in debian/rules 21:38:25 <liw> next question, please 21:38:36 <dapal> <merwok> QUESTION: Once I’ve made one or more packages you’d like to share, should I look into becoming a DM/DD/DMember? 21:38:45 <liw> yes, you should :) 21:39:01 <dapal> <bognu> QUESTION: this "upstream" thing is still confuing to me, cause I never used it. Upstream is provided by NOT debian people, right? So there is no format for this tarball? All it need is a Makefile? Or how is this defined 21:39:21 <liw> merwok, go look at the New Maintainer pages, and ask for feedback on debian-mentors 21:40:09 <merwok> Thanks liw. 21:40:16 <MadameZou> < bognu> QUESTION: this "upstream" thing is still 21:40:17 <MadameZou> confuing to me, cause I never used it. Upstream is 21:40:17 <MadameZou> provided by NOT debian people,right? So there is no 21:40:17 <MadameZou> format for this tarball? All it need is a Makefile? Or how is this defined 21:40:24 <dapal> MadameZou: FAIL :-) 21:40:40 <dapal> MadameZou: :* 21:40:43 <liw> bognu, upstream is the people who wrote the program originally, and not Debian, correct; simplifying a bit, Debian needs a tarball (if you have something else you need to convert); the tarball has no standard, but most sensible people have a ./confiugure script and a Makefile and that usually is enough 21:41:02 <MadameZou> oh god! dapal was faster! 21:41:10 <MadameZou> (sorry) 21:41:14 <liw> merwok, gregoa reminds me that another good way to learn is to join a packaging team, such as debian-perl, and I agree 21:41:18 <liw> next question, please 21:41:34 <dapal> <tixy> QUESTION: Isn't packaging the easy bit, what about maintaining it? (Bug triage, fixing, patch integration etc.) 21:41:46 <merwok> I’m still confused about DM vs. DD vs. DMember (the new thing) vs. in a team, but I’ll go to #d-m :) 21:42:30 <dapal> #endmeeting 21:42:35 <dapal> MadameZou: could you please? ↑ 21:42:38 <liw> wait 21:42:47 <dapal> ah, right 21:42:49 <liw> tixy, yes, you are quite correct; the initial packaging is the easy bit, and the other work is a lot more 21:42:49 <dapal> sorry liw 21:43:11 <liw> are there more questions waiting? if not, I could do with a break 21:44:00 <MadameZou> #endmeeting