These days I saw on Twitter a comment on how the behavior of the % (remainder) operator for negative numbers is weird in C.

Under what circumstances does someone actually *want* the C behavior, anyway (-5 % 3 == -2; 5 % -3 == 2) ?

— mcc (@mcclure111) April 6, 2015

I’ve seen this discussion come up numerous times in the Lua mailing list over the years. The reason being because Lua does it different, and most languages simply copy the behavior of C.

Today I saw Etiene’s cool demo of a mini JavaScript Duck Hunt clone that she presented at a “intro to programming” workshop for the Women in Leadership event in Bremen, Germany.

It’s a really nice demo of game behavior in a short span of code, and with the environment of Mozilla Thimble, it instantly enticed me to play around with the code and see what happened.

The first thing that came to my attention was that the ducks spawn at position x=0, and this made them “pop” into the screen. I thought that changing the initial value to something like x=-50 would be a small change to try and would produce a smoother effect (just change 0 to -50 in lines 56 and 116).

When I first tried that, the result was that they would show up, but wouldn’t start flapping their wings until they were at x=0. The reason is because the logic to switch sprites is made testing x % 30 for values 0, 10 and 20… and JavaScript’s % operator, like C’s, returns negative remainders for negative divisors.

My quick hack solution was to calculate

   var absx = Math.abs(this.x);

(which required me a visit to DuckDuckGo to figure out how to properly say “abs(x)” in JavaScript). This made the birds enter the screen flapping their wings. Yay!

Of course, this is not something you’d want to have to explain in an “intro to programming” workshop. It would be better if the animation “just worked” with that change…

But wait! If you have really sharp eyes, you’ll notice that from -50 to 0, the birds are flapping their wings upwards and from 0 on, they do it downwards. The animation is inverted!

The reason is because operating on abs(x) causes this:

Lua 5.3.0  Copyright (C) 1994-2015 Lua.org, PUC-Rio
> for i = -50, 100 do io.write(math.abs(i)%30, " ") end
20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 0 1 2 3 4 5 6 7 8 9 10

If I write a one-liner to simulate the sprite logic in Duck Hunt, I get this:

> for i = -50, 100 do r=math.abs(i)%30; io.write(r==0 and "1" or (r==10 and "2" or (r==20 and "3" or ".") ) ) end
3.........2.........1.........3.........2.........1.........2.........3.........1.........2.........3.........1.........2.........3.........1.........2

Indeed, it’s going 3,2,1, 3,2,1 at the negative numbers and then 1,2,3, 1,2,3 at the positive ones. But let’s just drop the math.abs in Lua and see what happens:

> for i = -50, 100 do r=i%30; io.write(r==0 and "1" or (r==10 and "2" or (r==20 and "3" or ".") ) ) end
2.........3.........1.........2.........3.........1.........2.........3.........1.........2.........3.........1.........2.........3.........1.........2

We get 1,2,3,1,2,3 all the way!

In my experience, the vast majority of times I used %, it was to tell something to “do this every X steps”, like Etiene does in her Duck Hunt. For this kind of purposes, I’m pretty convinced that Lua’s behavior for % is a lot better. It’s unfortunate that most other languages just decided to follow the example of C.

Of course, there are a million other ways to make the ducks flap their wings, with and without %, that’s not the point. But it intrigued me that, if JavaScript had Lua’s behavior for %, my initial tiny change would have “just worked”!

Here’s a diff between which columns are supported in Linux and FreeBSD ps commands, with aliases cleaned up by hand. The lists were gathered from their respective manpages. This is based on ps from FreeBSD 11.0 and from Linux procps-ng 3.3.9.

--- freebsd-ps-columns.txt	2015-03-15 23:44:58.710635450 -0300
+++ linux-ps-columns.txt	2015-03-15 23:42:43.894637864 -0300
@@ -1,81 +1,84 @@
 %cpu
 %mem
-acflag
 args
-class
+c
+cgroup
 comm
 command
-cow
-cpu
-dsiz
-emul
+eip
+esp
 etime
 etimes
-fib
+euser
+fgid
+fgroup
 flags
+fname
+fuid
+fuser
 gid
 group
-inblk
-jid
-jobc
-ktrace
+ipcns
 label
-lim
-lockname
-logname
 lstart
+lsession
 lwp
+machine
 majflt
 minflt
-msgrcv
-msgsnd
-mwchan
+mntns
+netns
 nice
-nivcsw
 nlwp
-nsigs
-nswap
-nvcsw
 nwchan
-oublk
-paddr
+ouid
 pgid
 pid
+pidns
+policy
 ppid
 pri
-re
+psr
 rgid
 rgroup
 rss
 rtprio
 ruid
 ruser
+sched
+seat
+sgi_p
+sgroup
 sid
 sig
 sigcatch
 sigignore
 sigmask
-sl
-ssiz
+size
+slice
+stackp
 start
+stat
 state
+supgid
+supgrp
+suser
 svgid
 svuid
-systime
-tdaddr
-tdev
+sz
+thcount
+tid
 time
+tname
 tpgid
-tsid
-tsiz
 tt
 tty
 ucomm
 uid
-upr
-uprocp
+unit
 user
-usertime
+userns
+utsns
+uunit
 vsz
 wchan
-xstat

UPDATE 2014-05-02: The fine folks at Bountysource extended the fundraiser for a few more weeks! Hurry up!

After more than 10 years developing free software (htop, GoboLinux, LuaRocks), this month I made my first organized attempt to get some funding for a project. Or rather, am making, since the fundraiser is still up at the time of this writing. But with 2 days to go and 22% of the funding goal met, I think it’s already a fine time to take a closer look at the experience and see what we can learn from it.

The idea, in short

htop is a text-mode process viewer. The backend code is Linux-specific, so unlike most free software, it’s not a portable project. Still, some people run it unmodified on FreeBSD (and GNU Hurd!) by using Linux emulation layers. There’s no such layer for Mac OSX, so one user ported htop by replacing its backend with a native implementation. There’s nothing wrong with that, of course — the source is open precisely to allow this kind of thing!

The problem is that, due to the way this fork was done, it couldn’t easily be merged back. So, it hasn’t kept up with the development and bugfixes of htop over the years. This hasn’t stopped it from making its way to Homebrew, through a series of forks. They’re all based on that version of htop 0.8.2 from five years ago, though. The result is that I keep getting bug reports from Mac users, some of them for very old bugs!

My time for free software is short as I usually take on lecturing and freelance jobs in addition to my PhD, so I though that, if there was enough interest, I could take on solving this situation as this season’s “freelance job”. I had just learned about BountySource through the neovim fundraiser so I thought “why not”, and created a fundraiser titled “Mac OS X support in the official htop 1.x tree”.

First contacts

I went through Github and searched for every fork of AndyA’s original htop-osx repository (when this fork was made my own htop repository was still in SourceForge, so his fork is not a “Github fork” of mine) and contacted every author to make them aware of my project. I also looked at the update history of the htop-osx formula in Homebrew and contacted everyone mentioned there. My goal was to reach everyone connected to htop-osx, let them know about the fundraiser (I didn’t want it to look like a “hostile fork” (even though I’m the original author!) ) and ask them how to better reach the OSX community.

Everyone was fine with the idea and nice to me, but unfortunately nobody had any tips on how to reach out to the audience. Fork authors suggested contacting Homebrew; Homebrew told me to contact the “htop-osx project”. Having had already contacted both, that didn’t take me far.

The general advice was to go through word-of-mouth. The thing is, I’m a Linux guy, and so I tend to gravitate around that universe in my online communities. I posted links on Twitter and I’ve got some pledges from friends, who were sure not that interested in the port but wanted to help out (thank you!). Then I posted in the htop mailing list, from which I also got some pledges, as shows of support.

Trying to reach wider audiences, I posted on a few Mac-related subreddits, where I got some pledges from /r/apple (and also on Hacker News, where it instantly tanked of course). After a couple hundred dollars in initial pledges the first few days, the fundraiser sat mostly dormant.

I wondered what else I could do to shake things up, but at the same time I couldn’t dedicate a lot of time to promote the fundraiser, between preparing classes and other commitments. This is a sign of a classic mistake: I didn’t do any planning beforehand. In particular, I was taken by surprise that once went through the BountySource fundraiser signup process, there was no option to set the duration of the campaign (yes, that means I didn’t read the fine print carefully; I guess I just skimmed it to see what was the percentage of their cut — 10%, by the way). It was only when I was up and running that I realized I only had one month to go. With only one month, I’d probably have set a lower goal.

BountySource has since remodeled its site to deemphasize “fundraisers” and focus more on developer teams and “bounties”, which are feature/bugfix requests for which one can set a value but, as I understand, can stay open with no time limit. I think that’s a better model.

An unexpected gift

Things were pretty quiet for a while, when I got a notification from none other than the original author of the Mac fork, AndyA. He got my email early on but could only respond a few weeks later. He apologized about the bug reports that were sent my way (mostly because htop’s segfault handler outputs a message directing the user to report bugs at the main htop website — something that none of the various Mac forks actually changed!) …and was extremely generous to pledge $500 to the fundraiser, which essentially doubled the amount I had raised at that point!

That was really kind of him, and it made me especially happy to know that my porting project had full support from the author of the original Mac fork. With that spur of motivation, I rushed to put to practice an idea I had a few days earlier. I thought of the best way I know to make the userbase of a free software project take notice, which is…

Making a release

htop is a mature piece of software, over ten years old by now. It had been over a year since the last release. There were some new features, bugfixes and performance improvements already sitting in its repository, that I had implemented and merged from contributors over the last year. So, I combed through the bugtracker fixing other pending issues and packed version 1.0.3.

I announced it at the usual places and made a post on /r/linux where it got a quite nice reception. That didn’t translate considerably into pledges, but at that point I didn’t really expect it would shift the fate of the fundraiser.

I even got a comment saying “I do not need a MacOSX version so I donated a few bucks on the regular site”.

What was learned

That comment above sums up a lot of what happened in this fundraiser. My userbase consists of Linux users, not Mac users, so they weren’t really engaged by this project.

Also, there were some fundamental problems with the idea of turning this Mac port into a fundraiser:

• even though I’m sure there are Mac terminal users out there (according to mentions of “htop on the Mac” on Twitter, and Mac reports in my bug tracker), I had no idea how to reach them in quantity, the way I could reach the Linux userbase.
• The whole issue with the existing Mac forks is that they crash occasionally (causing a slow but constant stream of bug reports my way). In other words, for most of the users, it works most of the time. Therefore, there is no pressing need to have something done about it.
• In the end, it’s a project to bring stability and maintainability, two qualities that aren’t really exciting.

With that in mind, I think the fundraiser would have had much better chances if it were aimed at Linux users and proposed cool features, such as

• Extensible custom meters for the header?
• Network measurements à la ntop?

Those are things I keep meaning to do at some point, but they require care in their implementation, in order to keep htop focused and lightweight.

What now

Even though the campaign hasn’t reached its funding goal, I think it has been a positive experience and a learning process (and the raised money is surely welcome, especially because there’s a particularly nice feeling in being compensated from developing free software). And at the very least, if I get new Mac bug reports in the future, I can point them to this and say “I tried!”

Aside from the mistakes listed above, I think the main thing I did right was contacting everyone related to the htop-osx fork. Not because the largest donation came from that, but because it opened a positive channel of communication with other developers, which will hopefully be fruitful for the project in the future.

With the current level of funding I will still have to take other freelance jobs this season, so I won’t have the time to produce a full Mac port. But the raised money will be used in sponsoring the development of the platform abstraction layer, which is the first step. Once I isolate the Linux-specific parts, it should be easier for the Mac development community (some of whom I was able to reach out in this project) to contribute a native backend and hopefully we’ll get the situation of htop on the Mac sorted out sooner rather than later.

UPDATE 2014-05-02: The fine folks at Bountysource extended the fundraiser for a few more weeks! Hurry up!

It’s been a while that I’ve been using Git comfortably — even though I still get stumped now and then, the workflow is much better than Subversion. I do finer grained commits with ease and I’m no longer afraid of making branches. Still, some projects of mine are still hosted in svn servers, mostly due to inertia and because they’re mostly single-person projects with the occasional patch and bug report but not much collaboration. For a more “social” project such as LuaRocks, git and Github have proven to be very useful tools.

Farewell to SourceForge

This, however, was the push I needed to move my projects away from SourceForge. From gimp.org:

In the past few months, we have received some complaints about the site where the GIMP installers for the Microsoft Windows platforms are hosted.

SourceForge, once a useful and trustworthy place to develop and host FLOSS applications, has faced a problem with the ads they allow on their sites - the green “Download here” buttons that appear on many, many adds leading to all kinds of unwanted utilities have been spotted there as well.

The tipping point was the introduction of their own SourceForge Installer software, which bundles third-party offers with Free Software packages. We do not want to support this kind of behavior, and have thus decided to abandon SourceForge.

From now on, Jernej Simončič, who provides the installer packages, uploads them to our FTP directly, and from there they will be distributed automatically to our mirrors. Please check Downloads page for updated information. http://gimp-win.sourceforge.net will remain active for the time being and direct users to the new download locations.

This saddens me a lot. I’ve been a SourceForge user for over 13 years. Having my first project there with a “.sourceforge.net” URL was a moment of pride; I felt I was becoming a free software developer “for real”, seeing my code there along with all those other projects I relied on daily, such as GIMP.

htop, in particular, has lived a beautiful life in SourceForge:

It blows my mind to think that these are direct downloads of the source code only. I assume the vast majority of users install htop running the distro package manager (apt-get, yum, etc.) but I have absolutely no way to estimate how many times this program has been installed. And that’s not only okay, it’s beautiful: it’s the nature of free software, the actual freedom, at work. There is no tight grip on users from a central authority, the code is roaming free in what is essentially a network of solidarity. I’ve certainly benefited from this network much more than I’ve contributed to it, but I’m happy to give a small part.

Another achievement I’m very proud of: a perfect 5-star rating score for htop. Surely not that many reviews when compared to really big projects, but it’s noteworthy to me at least, and I’m thankful to everyone who rated.

So, thanks for everything SourceForge, but it looks like it’s time to move on.

Say hello to Github, htop

I’ve already got a bunch of projects in Github so this change should not be traumatic.

I’m converting the repositories to Github and moving the website to my own domain. I’ve started with dit, which is a low-profile project, and everything went smoothly. So it’s time to do the same with htop.

The process is straightforward:

1. Here are the Instructions for importing from Subversion from Github
2. They recommend using svn2git. It took me a few tries to get the conversion perfect. I recommend taking a look at the generated repository with gitk and creating a file called ~/.svn2git/authors containing aliases matching your svn usernames with Github equivalents.

   My authors file looks like this:

   loderunner = Hisham Muhammad <hisham@gobolinux.org>
   hisham = Hisham Muhammad <hisham@gobolinux.org>
   

   And the command-line for svn2git:

   svn2git svn://svn.code.sf.net/p/htop/code/
   

3. The next step was to create a new Github repository. I did that through their web interface.
4. And then, to push the code from the locally generated git repo to Github:

   git remote add origin https://github.com/hishamhm/htop.git
   git push -u origin master
   

5. Next, I wanted to download the htop website from SourceForge. I usually edited things straight through their shell account, but this got me to an SFTP session in which I could fetch the files:

   sftp loderunner,htop@frs.sf.net

   (By the way, I actually use yafc, which I highly recommend, instead of sftp. Also, as I looked for its URL to post on this paragraph, I just learned that the original yafc 1.1.1 from 2001, which I still use and was hosted in SourceForge and sat there for years without updates, has been taken by a new group of developers who resurrected the project and develop it in Github.)

   The site for htop is really simple so moving it to a new location was a relatively quick process; just needed to grep all references to sf.net and svn, and get my own direct Paypal donation button (to replicate the one I had in SourceForge I used this and this).

6. Getting the archive of releases was a bit trickier. It’s also in frs.sf.net, but you need to know the correct directory. It doesn’t show up when you log in and ls. You have to cd to it:

   yafc ftp://loderunner@frs.sf.net
   # then, in the sftp session:
   cd /home/frs/project/htop
   get -p -r htop
   

   Now I have an archive of all releases, with their (mostly accurate) historical timestamps!

7. The final steps are shutting down the sf.net services, such as the bug tracker and the code repository. This can be done easily through their admin interface. I’ve never had any plans to migrate the bug tracker history; this simplifies things (and I don’t even know if it’s possible). I’m not shutting everything down immediately, though: until I make a proper new release and distro maintainers catch up with the new URLs, I’ll keep the download links active there.

   The original website, however, is now gone and replaced with the following:

   <html>
   <head>
   <meta http-equiv="refresh" content="0; url=http://hisham.hm/htop">
   </head>
   </html>
   

   (I did try an .htaccess file, but the sf.net servers kept redirecting to a default page even when I tried to do it via html… thanks to Lynx I realized it was redirecting with a 403 Forbidden error, so I removed the .htaccess file and things worked as expected.)

8. Still TODO: migrate the htop-general mailing list.

One criticism I’ve had of Github in the past was that it did not promote a culture of making proper releases like SourceForge always did, but that has improved in recent years. Github now has a “releases” feature which, while not perfect, does the job in many cases. I’m not sure if I’ll use it, since I prefer to make the tarballs for htop using the make dist feature from GNU Autotools. I hope to cause as little disruption as possible to the distro maintainers and I want to keep my packages looking the same.

The big test for the new setup will be the next release, which I hope to make in time for htop’s tenth anniversary(!), in the coming months. It is quite fitting that htop presents its new home in such a special occasion!

Today I had a Git dillema, which I posted about on Twitter:

git help! I’ve got two local branches (X and master) and their remote counterparts in github. What’s the right way to merge X into master?

Here’s the situation in more detail: I had a “feature branch” for the new luarocks doc command, and I published it on Github to get some feedback. So, at that point I had four branches: in my machine I had master and luarocks-doc. But I also had master and luarocks-doc in github as well. I’ve been pushing master into origin/master and luarocks-doc into origin/luarocks-doc… I thought about doing a straightforward local merge but history could get messy and people would complain I didn’t do it right.

Alexander Gladysh came to the rescue, with the following sequence, which he listed to me through IM and worked flawlessly:

git fetch
git checkout master
git merge --ff-only origin/master
git checkout luarocks-doc
git merge --ff-only origin/luarocks-doc
git rebase master
git checkout master
git merge --ff-only luarocks-doc
git push origin master

As he said, “then you want to either update or delete luarocks-doc in the origin”:

To update:

git push -f origin luarocks-doc

To delete:

git push origin :luarocks-doc

The –ff-only flag tells merge not to perform incomplete merges (as it happened to me so often!) so it only runs it if the command would perform its task completely.

Alexander also gave me some links to material on Git workflow models, which I’ll be sure to read:

• A successful Git branching model
• A list of popular workflow models

Hope that helps you as Alexander helped me. Sharing it forward!