hisham hm

Writing release announcement emails

Mailing lists are not exactly fashionable nowadays, but some of them remain relevant for some communities. The Lua community is one such example. As of 2017, a lot of what goes on in the Lua module development world still resonates in lua-l. With over 2500 subscribers, it’s a good way to kickstart interest in your new project.

Mailing list users tend to be somewhat pedantic about etiquette guidelines for posting, especially for announcements and the like. So, I usually follow this little formula for writing release announcement emails, which has been effective for me:

An example of an upgrade announcement is here:

[ANN] LuaRocks 2.4.2

Hello, list!

I'm happy to announce LuaRocks 2.4.2. LuaRocks is the Lua package
manager. (For more information, please visit http://luarocks.org )

http://luarocks.org/releases/luarocks-2.4.2.tar.gz
http://luarocks.org/releases/luarocks-2.4.2-win32.zip

Those of you on Unix who are running LuaRocks as a rock (i.e. those
who previously installed using `make bootstrap`) can install it using:

   luarocks install luarocks

What's new since 2.4.1:

* Fixed conflict resolution on deploy/delete
* Improved dependency check messages
* Performance improvements when removing packages
* Support user-defined `platforms` array in config file
* Improvements in Lua interpreter version detection in Unix configure script
* Relaxed Lua version detection to improve support for alternative
implementations (e.g. Ravi)
* Plus assorted bugfixes and improvements

This release contains commits by Peter Melnichenko, Robert Karasek and myself.

As always, all kinds of feedback is greatly appreciated.

Thank you, enjoy!

-- Hisham

An example of a new project announcement is here:

[ANN] safer - Paranoid Lua programming

Hi,

Announcing yet another "strict-mode" style module: "safer".

* http://github.com/hishamhm/safer

Install with
   luarocks install safer

# Safer - Paranoid Lua programming

Taking defensive programming to the next level. Use this module
to avoid unexpected globals creeping up in your code, and stopping
sub-modules from fiddling with fields of tables as you pass them
around.

## API

#### `safer.globals([exception_globals], [exception_nils])`

No new globals after this point.

`exception_globals` is an optional set (keys are strings, values are
`true`) specifying names to be exceptionally accepted as new globals.
Use this in case you have to declare a legacy module that declares a
global, for example. A few legacy modules are already handled by
default.

`exception_nils` is an optional set (keys are strings, values are
`true`) specifying names
to be exceptionally accepted to be accessed as nonexisting globals.
Use this in case code does feature-testing based on checking the
presence of globals. A few common feature-test nils such as `jit` and
`unpack` are already handled by default.

#### `t = safer.table(t)`

Block creation of new fields in this table.

#### `t = safer.readonly(t)`

Make table read-only: block creation of new fields in this table
and setting new values to existing fields.

Note that both `safer.table` and `safer.readonly` are implemented
creating a proxy table, so:

* Equality tests will fail: `safer.readonly(t) ~= t`
* If anyone still has a reference to this table prior
  to creating the safer version, they can still mess
  with the unsafe table and affect the safe one.

About
-----

Licensed under the terms of the MIT License, the same as Lua.

During its genesis, this module was called "safe", but I renamed it
to "safer" to remind us that we are never fully safe. ;)

-- Hisham
http://hisham.hm/ - @hisham_hm

Hope this helps!


Fun hack to redirect stdout and stderr in order

Prologue

This is anecdote about roundabout ways to get stuff done. Pierre mentioned in the comments below that a proper way to solve this is to use unbuffer (though it does _not_ produce the exact same order as the terminal!). But if you want to read the improper way to do this, read on! :)

The story

Due to buffering, the terminal messes with the order of stdout and stderr of a program when redirecting to a file or another program. It prints the outputs of both descriptors in correct order relative to each other when printing straight to the terminal:

] ./my_program
stdout line 1
stdout line 2
stderr line 1
stdout line 3
stderr line 2
stderr line 3

This doesn’t change the order:

] ./my_program 2>&1
stdout line 1
stdout line 2
stderr line 1
stdout line 3
stderr line 2
stderr line 3

but it changes the order when saving to a file or redirecting to any program:

] ./my_program 2>&1 | cat
stderr line 1
stderr line 2
stderr line 3
stdout line 1
stdout line 2
stdout line 3

This behavior is the same in three shells I tested (bash, zsh, dash).

A weird “solution”

I wanted to save the log while preserving the order of events. So I ended up with this evil hack:

] strace -ewrite -o trace.txt -s 2048 ./my_program; sed 's,^[^"]*"\(.*\)"[^"]*$,\1,g;s,\\n,,g;' trace.txt > mytrace.txt
] cat mytrace.txt
stdout line 1
stdout line 2
stderr line 1
stdout line 3
stderr line 2
stderr line 3
+++ exited with 0 +++

It turns out that strace does log each write in the correct order, so I’m catching the write syscall.

Note the limitations: it truncates lines to 2048 characters (good enough for my logs) and I was simply cutting off n and not cleaning up any other escape characters. But it worked well enough so I could read my ordered logs in a text editor!


Fixing permission errors for scanning on Linux: running XSane as a regular user

I decided to stop running XSane as root. Here’s what I had to do:

* Restored the ownership of ~/.sane back to my user: sudo chown -R hisham ~/.sane
* Made /var/lock/sane writable by a regular user: sudo chmod a+rwxt /var/lock/sane
* As a quick-and-dirty fix, as suggested here, I gave permissions to my USB devices: sudo chmod a+rw -R /dev/bus/usb/*
* For a more permanent (and still untested) fix, I added a 40-libsane.rules file (found here) in /etc/udev/rules.d. This file already includes the USB id for my scanner. You may have to add yours.


First time playing with OpenBSD

A list of notes on my first experience with OpenBSD:

* It wasn’t obvious at first which file to download, navigating their FTP site. I ended up with cd58.iso, which was the right choice. A tiny 7.8MB ISO!
* As soon as I typed “OpenBSD” as the VM name in VirtualBox, it gave me OpenBSD defaults. It defaulted to 64MB of RAM (only!), but I chose 256M, and the default 2G disk image.
* The first boot gave me a few options, including “Install” and “Autoinstall”. I chose “Autoinstall” since I thought that would install with the defaults, but it looked for an installation script in the local network (and obviously didn’t find it). Reboot, “Install” and off we go.
* No Dvorak options in the list of keyboards in the installer.
* I went with the default options and installed all offered packages (it offered me a list of coarse-grained bundles). It asked me about creating users, partitions, configuring network and if I wanted to run sshd by default.
* To enable Dvorak, the internet told me to do kbd us.dvorak nice.
* To make it permanent, I just grepped /etc for kbd and found out that cat us.dvorak > /etc/kbdtype would suffice.
* To install things, become root and then use pkg_add. For example: pkg_add wget
* By default it includes df but not free.
* To get color in the terminal, set TERM=wsvt25. htop showed in its usual colors!
* The default PATH includes the current (.) directory! Take that, Linux status quo!
* iconv is installed under /usr/local, and its library exports symbols with names such as libiconv_open instead of the usual iconv_open, which fools the typical AC_CHECK_LIB test in Autoconf. (In the source code, iconv_open works, so I guess iconv.h uses #define to translate the name. Added a hack to Dit to make it build cleanly out-of-the-box.


String interpolation in Lua

Lua is known for having a very lean standard library, and for providing mechanisms to do things instead of a ton of features.

String interpolation isn’t available out of the box, but doing it in Lua isn’t a new trick. In fact, the manual includes it as an example of string.gsub:

local t = {name="lua", version="5.3"}
x = string.gsub("$name-$version.tar.gz", "%$(%w+)", t)
--> x="lua-5.3.tar.gz"

This applies to members of a table only, though. Python is introducing a general string-interpolation syntax:

a = "Hello"
b = "World"
f"{a} {b}"
f"{a + ' ' + b}"

Given that Lua supports the f"str" syntax for functions with a single string argument, I thought it would be nice to put its Lua-provides-the-mechanisms ethos to test by trying to write my own Python-like f-string formatter.

And here it is, in all its 28-line glory (and I went for readability, and not to write it as short as possible):

function f(str)
   local outer_env = _ENV
   return (str:gsub("%b{}", function(block)
      local code = block:match("{(.*)}")
      local exp_env = {}
      setmetatable(exp_env, { __index = function(_, k)
         local stack_level = 5
         while debug.getinfo(stack_level, "") ~= nil do
            local i = 1
            repeat
               local name, value = debug.getlocal(stack_level, i)
               if name == k then
                  return value
               end
               i = i + 1
            until name == nil
            stack_level = stack_level + 1
         end
         return rawget(outer_env, k)
      end })
      local fn, err = load("return "..code, "expression `"..code.."`", "t", exp_env)
      if fn then
         return tostring(fn())
      else
         error(err, 0)
      end
   end))
end

It works just like the Python example:

a = "Hello"
b = "World"
print(f"{a} {b}")

Unlike the one-liner from the Lua manual, it also works with local variables:

local c = "Hello"
local d = "World"
print(f"Also works with locals: {c} {d}")

do
   local h = "Hello"
   do
      local w = "World"
      print(f"Of any scope level: {h} {w}")
   end
end

Some more demos:

print(f"Allows arbitrary expressions: one plus one is {1 + 1}")

local t = { foo = "bar" }
print(f"And values: t.foo is {t.foo}; print function is {_G.print}")

local ok, err = pcall(function()
   print(f"This fails: { 1 + } ")
end)
print("Errors display nicely: ", err)

If there’s interest, I can make this a module in LuaRocks (probably calling it F rather than f).

Update! This is now available in LuaRocks as a module! Install it with:

luarocks install f-strings

More info at the f-strings GitHub page. Enjoy!