hisham hm

How to write Lua modules in a post-module() world

Our beloved function module() is really going away. As of Lua 5.2 it’s only available with compatibility flags on, and the writing’s on the wall: it is going away for good in Lua 5.3. So, in a new Lua project I wrote this past semester, I decided to write it without using module(), while making sure my code runs on both Lua 5.1 and 5.2 (as a side result, I started the compat52 project, which allows you to write code in a well-behaved Lua 5.2 style and make it run on both 5.1 and 5.2).

So, why I liked module() in the first place? While vilified by some, I think the pros of module() largely trumped its cons. It had indeed some nice properties:

So, how to try to retain some of these properties without module()? The solution I found was to adopt some bits of policy, which I list below.

Yes, bits of policy. I know many in the Lua world hate policies, but of course I’m not putting a gun against anyone’s head to follow them. I’m only sharing what works for me and hopefully you may find some use. And don’t worry it’s nothing too esoteric, and it’s mostly cherry-picking some established practices.

Starting from the outside in

Keeping in mind that the goal of a module is to be required by client code, this is how a module foo.bar will be used:

local bar = require("foo.bar") -- requiring the module

bar.say("hello") -- using the module

An interesting observation here is that although we have a hierarchical structure of modules, the practice of loading them into locals means that in use they have to be accomodated in a flat namespace. So here’s Policy Bit #1:

Policy Bit #1: always require a module into a local named after the last component of the module’s full name.

Don’t do stuff such as local skt = require("socket") — code is much harder to read if we have to keep going back to the top to check how you chose to call a module.

Naming modules

Now that you know that your module will end up in people’s locals, please take that into consideration when naming your module. (I wish we had a capitalization policy to separate that nicely, but naming things LikeThis in Lua tends to be used only for object-oriented code.)

The idea is to choose a name that strikes a balance between convenience and uniqueness, and that is usable. And what better way to achieve this other than using this name. So, here’s Policy Bit #2, let’s use the module name in its declaration!

Policy Bit #2: start a module by declaring its table using the same all-lowercase local name that will be used to require it.

So, in the beginning of module foo.bar (which will live in foo/bar.lua), we begin with:

local bar = {}

It’s not a nice self-documenting header as we used to have with module("foo.bar", package.seall), but it’s something. We can improve that with LDoc comments:

--- @module foo.bar
local bar = {}

Don’t name your module something like “size”.

Declaring functions

When I’m scrolling through source code, I like to be able to tell what’s the sphere of influence of the piece of code I’m looking at. Is this a tiny helper function that’s only used below this line in this file? Is it an important function that’s used by other clients, so that an added or removed argument would mean API breakage? Ideally I like to be able to tell that without running back and forth in the code, so I really like visibility to be explicit in the syntax.

We must not declare global functions (or globals of any type, really!) in our modules, so using “globals vs. locals” to tell the difference won’t cut it. We have some alternatives, though. But first, let’s assert one thing:

Policy Bit #3: Use local function to declare local functions only: that is, functions that won’t be accessible from outside the module.

That is, local function helper_foo() means that helper_foo is really local.

This sounds obvious, but there are advocates of declaring all functions, public and private, as local functions and then writing an “export list” at the bottom of the module. Reading code written like this feels to me like a thriller with a twist ending: “haha, I was a public function all along!”

How to write public functions then? We must not declare global functions, but there are alternatives. Say we’re writing a function that will be used in client code as bar.say("hello"). It’s nice that we can declare it just like that:

function bar.say(greeting)
   print(greeting)
end

Policy Bit #4: public functions are declared in the module table, with dot syntax.

Visibility is made explicit through syntax. This is the same idea advocated by those who tell you to name your module tables “M”, except that you’re eating your own dogfood and using the name you expect your users to use. It’s also more consistent, since calls of say() are written bar.say() everywhere, instead of say(), M.say(), etc. (Also, “M.” looks really really ugly and people can’t decide if they want to use “M” or “_M”.)

In case you have speed concerns about having your calls go through the module table: first, this is what your users will go through; second, this is no different than using colon-syntax and dispatching through self and nobody complains about that; third, if you really need it (and have a benchmarked case for it), sure go ahead and make locals for optimization clearly marked as such; fourth, if you’re really after speed you’re probably using LuaJIT and last I heard the value of caching functions into locals is put into question there.

Classes and objects

When talking about classes and objects, it’s then time to talk about things named LikeThis. (If you don’t do OOP, feel free to skip this section!)

As we did above, let’s start look from the outside in: how to instantiate an object. There are two common practices (oh why I am not surprised :( )… you either make a class table with a “new” method, or make the “class object” callable (as a function or a table with a __call metamethod — wait, that makes it three practices…)

local myset1 = Set.new() -- style 1
local myset2 = Set() -- style 2.1 (set is a function)
local myset3 = Set() -- style 2.2 (set is a table)

If your module represents a class, I tend to like style 1 better because:

If all your module does is define a class, I guess it makes sense to name the module file MyClass.lua and have the class table be the module table. But I prefer not to do that, because often what we store as “static” class methods in purely OOP languages are really module functions. I still use the uppercase table when implementing the class, like this:

--- @module myproject.myclass
local myclass = {}

-- class table
local MyClass = {}

function MyClass:some_method()
   -- code
end

function MyClass:another_one()
   self:some_method()
   -- more code
end

function myclass.new()
   local self = {}
   setmetatable(self, { __index = MyClass })
   return self
end

return myclass

It’s easy to see in the code above that the functions with MyClass in their signature are methods. Sometimes it’s nice to declare the functions as fields inside the table declaration, but declaring methods separately as in the example above allows you to keep local helper functions closer to where they’re used.

If all the module does is declare the class, the class and module table may be one and the same. If you want to use style 2, we get something like this:

--- @module myproject.MyClass
local MyClass = {}

function MyClass:some_method()
   -- code
end

function MyClass:another_one()
   self:some_method()
   -- more code
end

local metatable = {
   __call = function()
      local self = {}
      setmetatable(self, { __index = MyClass })
      return self
   end
}
setmetatable(MyClass, metatable)

return MyClass

Both methods are acceptable, as long as it’s easy and obvious to tell you’re doing OOP:

Policy Bit #5: construct a table for your class and name it LikeThis so we know your table is a class.

Policy Bit #6: functions that are supposed to be used as object methods should be clearly marked as such, and the colon syntax is a great way to do it.

Don’t make people reading your function have to guess (or look up) if it is a method, a public module function or a local function.

Wrapping up

Return the module table. It’s a bit of boilerplate, but it’s what we have to deal with in a module()less world:

return bar

Policy Bit #7: do not set any globals in your module and always return a table in the end.

To sum it all up, a complete module foo.bar would look like this:

--- @module foo.bar
local bar = {}

local function happy_greet(greeting)
   print(greeting.."!!!! :-D")
end

function bar.say(greeting)
   happy_greet(greeting)
end

return bar

The result is that we type a bit more than we did with module(), and we risk polluting the global namespace if we’re not careful, but with this set of policies, we have:

…which mostly matches what I liked about module(), to the extent that can be done without _ENV tricks.

I’ve been using these policies successfully in a university project, and my plan is to follow them when I update the LuaRocks codebase to drop the use of module(). Consider your self encouraged to adopt some or hopefully all of them, but most importantly, whatever you do, be consistent! Good luck in this brave post-module() world!


  1. Roland_Y

    Friday, January 3, 2014 - 13:19:32

    Hi,

    Very nice article! I learned a few points, and in general, I like those policies which are in favor of clean code. I have bookmarked this, so that I can mention the articles on some other places as complementary reading.

    I have a few questions:

    Q1. About classes, why in the class constructor do you set as a metatable to the returned instance an “anonymous” table (with an __index pointing to the “class” itself) ?
    Actually, I do the following most of the time:

    local MyClass = {}
    MyClass.__index = MyClass
    function MyClass:new(…)
    return setmetatable({…}, MyClass)
    end

    And well, in case I need to support inheritance:

    local MyClass = {}
    function MyClass:new(…)
    self.__index = self
    return setmetatable({…}, self)
    end

    I am not saying this is better, but since this is my general pattern, I would be very grateful if you can point out some potential problems with this notation, it’ll be much of help to me to improve my code.

    Q2. I have in mind another way of building classes, which is a mix between … closures and upvalues. Just like:

    local SomeClass = {}
    function SomeClass.new(attr1, attr2)
    local self = {}
    attr1 = attr1 or
    attr2 = attr2 or

    function self.some_method(…) … end
    function self.setAttr1(value) attr1 = value end
    function self.getAttr2() return attr2 end
    return self
    end

    As you can see, this one seems does not make use of (get/set)metatable, since everything is defined in the returned object itself. It allows full privacy, too, since attributes cannot be reached indexing in the object itself.
    Do you have some specific thoughts about that ?

    That’s it, thanks in advance.

  2. Roland_Y

    Friday, January 3, 2014 - 13:46:36

    And ah, I forgot to mention another article from Enrique Garçia I came accross a while ago, that you might to check out too. It also defines some policies on dealing with OOP and privacy with Lua, though it is tight to a specific library, named MiddleClass.
    It is a very nice article, there you go: https://github.com/kikito/middleclass/wiki/Private-stuff

    Thanks again.

  3. hisham

    Friday, January 3, 2014 - 19:16:45

    Thanks for the comments!

    I wasn’t really advocating one particular style of OOP over others. (I even thought a bit if I should have added a concrete example or not because I was afraid it would stray me from the subject of modules.) I don’t think there is a “one better way” to do OOP in Lua; each of them have their pros and cons. That’s why I tried to keep my “policy bits” in the OOP section really general: basically, use TitleCase in class objects so we all know it’s a class and when declaring methods, make it obvious that the function is a method. Both your examples and the code from Enrique’s middleclass already do this.

    On the other hand, in Pierre’s code (his last code snippet), we can tell that the functions are methods because of the use of self (so it sort of follows Policy #6). But reading his function we don’t know what class the method belongs to, and reading the whole file we don’t even know the name of the class. Of course, if the file contains the definition of a single class and it is called MyClass.lua, then both questions are answered. These are nice things to do by the way, but it’s harder to spot at first glance. When reading a longer module written in his style, even in MyClass.lua, how do you know that the function is not part of another internal helper class? You’ll only figure out when you find the constructor function in the end.

    I agree with him that the final bit is the most important — in the end, interfaces are indeed more important than implementation!

    On your specific questions:

    Q1 is totally fine by me. Creating a new mt for each instance as I did there was a bit wasteful, I admit. When I have a bunch of metamethods (__eq, __lt, etc) I tend to write a separate mt table and have all objects share that. Other times I actually copied over the methods to the instance tables, to avoid the metatable lookup. It all depends on the situation.

    On Q2, achieving real privacy is always iffy in languages such as Lua, see Pierre’s comments about the limited effectiveness of preventing monkey patching. Nowadays I don’t jump extra hoops to stop users from accessing attributes; for example, in your constructor, even if they can’t access the attribute, they can still replace the getter and the setter altogether! My concern about functions vs. local functions has to do more with having well-defined interfaces (to help me know what affects what in the code) than to actually prevent people from being able to call the functions.

Add comment

Fill out the form below to add your own comments.

CAPTCHA imageReload imageAudible version of CAPTCHA-image