LFE MACHINE MANUAL

Adatped from multiple sources
by Duncan McGreggor and Robert Virding

Published by Cowboys 'N' Beans Books

https://github.com/cnbbooks ◈ http://cnbb.pub/ ◈ info@cnbb.pub

First electronic edition published: 2020

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

About the Cover

The LFE "Chinenual" cover is based upon the Lisp Machine Manual covers of the early 80s. The Lisp Machine Manual editions we have seen from 1977 and 1979 had only hand-typed title pages, no covers, so we're not sure if the famous graphic/typographic design occurred any earlier than 1982. We've also been unable to discover who the original designer was, but would love to give them credit, should we find out.

The Original

The Software Preservation Group has this image on their site:

Bitsavers has a 3rd edition of the Chinenual with the full cover

The LFE Edition

Whole Cover

Back Cover

The Spine

Dedication

To all LFE Community members, Lispers, programmers as well as all our friends and families.

Preface

The original Lisp Machine Manual, the direct spiritiaul ancestor of the LFE Machine Manual, described both the language and the "operating system" of the Lisp Machine. The language was a dialect of Lisp called Zetalisp. Zetalisp was a direct descendant of MACLISP, created as a systems programming language for the MIT Lisp machines. This is of special note since Erlang was created as a systems programming language too. One of it's co-creators, Robert Virding, created Lisp Flavoured Erlang (LFE) based upon his expereinces with Franz Lisp (which based largely upon MACLISP), Portable Standard Lisp (itself an experiment in systems programming), and ultimately in an implementation he made of Lisp Machine Flavors on top of VAX/VMS where he extensively utilized the Lisp Machine Manual.

As such, LFE has a very strong inheritance of systems programming from both parents, as it were. First and foremost, it is a BEAM language written on top of the Erlang VM and from which it strays very little. Secondly, it is a Lisp dialect. It is, however, entirely a systems programming language.

Which brings us back to Zetalisp and the Lisp Machine Manual. It seemed only fitting to base the LFE manual upon the fantastic work and docuentation that was done on Lisp systems programming in the 70s and 80s, work that so many of us treasure and adore and to which we still defer. Thus the machine that is OTP in the context and syntax of the LFE Lisp dialect is extensively documented in the LFE MACHINE MANUAL.

Forward

The original Lisp programming language was implemented nearly 65 years ago¹ at MIT (Massachusetts Institute of Technology), as part of the work done by John McCarthy and Marvin Minsky at the then-nascent AI (Artificial Intelligence) Lab. In the intervening half-century, the original Lisp evolved and experienced significant transformation in a technological diaspora originally fuelled by an explosion of research in the field of artificial intelligence. Through this, the industry witnessed 40 years of development where countless independent Lisp implementations were created. A small sampling of these number such favourites as Lisp 1.5, MacLisp, ZetaLisp, Scheme, Common Lisp, and ISLISP. However, the early 1990s saw the beginning of what would become the AI winter, and Lisp declined into obscurity – even notoriety – as graduating computer scientists were ushered into the “enterprise” world of Java, never to look back.

Except some did. By the early to mid-2000s, groundwork was being laid for what is now being recognized as a “Lisp renaissance.” The rediscovery of Lisp in the new millennium has led to the creation of a whole new collection of dialects, many implemented on top of other languages: Clojure on Java; LFE (Lisp Flavoured Erlang) and Joxa on Erlang; Hy on Python; Gherkin on Bash. New languages such as Go and Rust have several Lisp implementations, while JavaScript seems to gain a new Lisp every few years. While all of these ultimately owe their existence to the first Lisp, conceived in 1956² and defined in 1958³, they represent a new breed and a new techno-ecological niche for Lisp: bringing the power of meta-programming and the clarity of near syntaxlessness⁴ to established language platforms. Whereas in the past Lisp has been an either-or choice, the new era of Lisps represents a symbiotic relationship between Lisp and the language platforms or VMs (virtual machines) upon which they are implemented; you now can have both, without leaving behind the accumulated experiences and comfort of your preferred platform.

Just as the Lisp implementations of the 1960s were greatly impacted by the advent of time-sharing computer systems, the new Lisps mentioned above like Clojure and LFE have been deeply influenced not only by their underlying virtual machines, but – more importantly – by the world view which the creators and maintainers of those VMs espoused. For the Erlang ecosystem of BEAM (Bogdan/Björn's Erlang Abstract Machine) languages, the dominant world view is the primacy of highly-concurrent, fault-tolerant, soft real-time, distributed systems. Erlang was created with these requirements in mind, and LFE inherits this in full. As such, LFE is more than a new Lisp; it is a language of power designed for creating robust services and systems. This point bears some discussion in order to properly prepare the intrepid programming language enthusiast who wishes to travel through the dimensions of LFE.

Unlike languages whose prototypical users were developers working in an interactive shell engaged in such tasks as solving math problems, Erlang's prototypical “user” was a telecommunications device in a world where downtime was simply unacceptable. As such, Erlang's requirements and constraints were very unusual when compared to most other programming languages of its generation.⁵ Erlang, and thus its constellation of dialects, was designed from the start to be a programming language for building distributed systems, one where applications created with it could survive network and systems catastrophes, millions of processes could be sustained on a single machine, where tens and hundreds of thousands of simultaneous network connections could be supported, where even a live, production deployment could have its code updated without downtime. Such is the backdrop against which the Erlang side of the LFE story unfolds – not only in the teaching and learning of it, but in its day-to-day use, and over time, in the minds of its developers.

When bringing new developers up to speed, this perspective is often overlooked or set aside for later. This is often done intentionally, since one doesn't want to overwhelm or discourage a newcomer by throwing them into the “deep end” of distributed systems theory and the critical minutia of reliability. However, if we ignore the strengths of LFE when teaching it, we do our members as well as ourselves a disservice that leads to much misunderstanding and frustration: “Why is LFE so different? You can do X so much more simply in language Y”. It should be stated quite clearly in all introductory materials that the BEAM languages are not like other programming languages; in many ways, the less you rely upon your previous experiences with C, Java, Python, Ruby, etc., the better off you will be.

When compared to mainstream programming languages, Erlang's development is akin to the divergent evolution of animals on a continent which has been completely isolated for hundreds of millions of years. For instance, programming languages have their “Hello, world”s and their “first project”s. These are like lap dogs for newcomers to the programming world, a distant and domesticated version of their far more powerful ancestors. Though each language has its own species of puppy to help new users, they are all loyal and faithful companions which share a high percentage of common genetic history: this is how you print a line, this is how you create a new project. Erlang – the Down Under of programming languages – has its “Hello, world”s and “first project”s, too. But in this case, the lapdog does not count the wolf in its ancestral line. It's not even a canid.⁶ It's a thylacine⁷ with terrifying jaws and and unfamiliar behaviours. Its “hello world” is sending messages to thousands of distributed peers and to nodes in supervised, monitored hierarchies. It has just enough familiarity to leave one feeling mystified by the differences and with the understanding that one is in the presence of something mostly alien.

That is the proper context for learning Erlang when coming from another programming language.

In LFE, we take that a step further by adding Lisp to the mix, supplementing a distributed programming language platform with the innovation laboratory that is Lisp. Far from making the learning process more difficult, this algebraic, time-honoured syntax provides an anchoring point, a home base for future exploration: it is pervasive and familiar, with very few syntactical rules to remember. We have even had reports of developers more easily learning Erlang via LFE.

In summary, by the end of this book we hope to have opened the reader's eyes to a new world of promise: distributed systems programming with a distinctly 1950s flavour – and a capacity to create applications that will thrive for another 50 years in our continually growing, decentralized technological world.

Duncan McGreggor
2015, Lakefield, MN &
2023, Sleepy Eye, MN

Notes

¹ The first draft of this forward which was written in 2015 said "almost 60 years ago" but was never published. Today, at the end of 2023, this content is finally seeing the light of day, with the origins of Lisp receding further into the past ...

² See McCarthy's 1979 paper History of Lisp, in particular the section entitled “LISP prehistory - Summer 1956 through Summer 1958”.

³ Ibid., section “The implementation of LISP”.

⁴ As you learn and then take advantage of Lisp's power, you will find yourself regularly creating new favourite features that LFE lacks. The author has gotten so used to this capability that he has applied this freedom to other areas of life. He hopes that you can forgive the occasional English language hack.

⁵ In fact, in the 1980s when Erlang was born, these features were completely unheard of in mainstream languages. Even today, the combination of features Erlang/OTP (Open Telecom Platform) provides is rare; an argument can be made that Erlang (including its dialects) is still the only language which provides them all.

⁶ The family of mammals that includes dogs, wolves, foxes, and jackals, among others.

⁷ An extinct apex predator and marsupial also known as the Tasmanian tiger or Tasmanian wolf.

Acknowledgments

TBD

[gonna be a long list ...]

Part I - Getting Started

Introduction

Far out in the uncharted backwaters of the unfashionable end of computer science known as "distributed systems programming" lies a small red e. Orbitting this at a distance roughly proportional to the inverse of the likelihood of it being noticed is an utterly insignificant little green mug filled with the morning beverage stimulant equivalent of That Old Janx Spirit. Upon that liquid floats a little yellow 𝛌 whose adherents are so amazingly primitive that they still think conss, car, and cdr are pretty neat ideas.

This is their book.

Their language, Lisp Flavoured Erlang (henceforth "LFE"), lets you use the archaic and much-beloved S-expressions to write some of the most advanced software on the planet. LFE is a general-purpose, concurrent, functional Lisp whose underlying virtual machine (Erlang) was designed to create distributed, fault-tolerant, soft-realtime, highly-availale, always-up, hot-swappable appliances, applications, and services. In addition to fashionable digital watches, LFE sports immutable data, pattern-matching, eager evaluation, and dynamic typing.

This manual will not only teach you what all of that means and why you want it in your breakfast cereal, but also: how to create LFE programs; what exactly are the basic elements of the language; the ins-and-outs of extremely efficient and beautiful clients and servers; and much, much more.

Note, however, that the first chapter is a little different than most other books, and is in fact different from the rest of the chapters in this manual. We wrote this chapter with two guiding thoughts: firstly and foremost, we wanted to provide some practical examples of code-in-action as a context in which a programmer new to LFE could continuously refer -- from the very beginning through to the successful end -- while learning the general principles of that language; secondly, most programming language manuals are dry references to individual pieces of a language or system, not representatives of the whole, and we wanted to provide a starting place for those who learn well via examples, who would benefit from a view toward that whole. For those who have already seen plenty of LFE examples and would rather get to down to the nitty-gritty, rest assured we desire your experience to be guilt-free and thus encourage you to jump into next chapter immediately!

This book is divided into 6 parts with the following foci:

Introductory material
Core data types and capabilities
The basics of LFE code and projects
Advanced language features
The machine that is OTP
Concluding thoughts and summaries

There is a lot of material in this book, so just take it section by section. If at any time you feel overwhelmed, simply set down the book, take a deep breath, fix yourself a cuppa, and don't panic.

Welcome to the LFE MACHINE MANUAL, the definitive LFE reference.

About LFE

LFE is a modern programming language of two lineages, as indicated by the expansion of its acronym: Lisp Flavoured Erlang. In this book we aim to provide the reader with a comprehensive reference for LFE and will therefore explore both parental lines. Though the two language branches which ultimately merged in LFE are separated by nearly 30 years, and while LFE was created another 20 years after that, our story of their origins reveals that age simply doesn't matter. More significantly, LFE has unified Lisp and Erlang with a grace that is both simple and natural. This chapter is a historical overview on how Lisp, Erlang, and ultimately LFE came to be, thus providing a broad context for a complete learning experience.

What Is LFE?

LFE is a Lisp dialect which is heavily flavoured by the programming language virtual machine upon which it rests, the Erlang VM.¹ Lisps are members of a programming language family whose typical distinguishing visual characteristic is the heavy use of parentheses and what is called prefix notation.² To give you a visual sense of the LFE language, here is some example code:

(defun remsp
  (('())
   '())
  (((cons #\  tail))
   (remsp tail))
  (((cons head tail))
   (cons head (remsp tail))))

This function removes spaces from a string that it passed to it. We will postpone explanation and analysis of this code, but in a few chapters you will have the knowledge necessary to understand this bit of LFE.

Besides the parentheses and prefix notation, other substantial features which LFE shares with Lisp languages include the interchangeability of data with code and the ability to write code which generates new code using the same syntax as the rest of the language. Examples of other Lisps include Common Lisp, Scheme, and Clojure.

Erlang, on the other hand, is a language inspired most significantly by Prolog and whose virtual machine supports not only Erlang and LFE, but also newer BEAM languages including Joxa, Elixir, and Erlog. BEAM languages tend to focus on the key characteristics of their shared VM: fault-tolerance, massive scalability, and the ability to easily build soft real-time systems.

One way of describing LFE is as a programming language which unites these two. LFE is a happy mix of the serious, fault-tolerant philosophy of Erlang combined with the flexibility and extensibility offered by Lisp dialects. It is a homoiconic distributed systems programming language with Lisp macros which you will soon come to know in much greater detail.

A Brief History

To more fully understand the nature of LFE, we need to know more about Lisp and Erlang – how they came to be and even more importantly, how they are used today. To do this we will cast our view back in time: first we'll look at Lisp, then we'll review the circumstances of Erlang's genesis, and finally conclude the section with LFE's beginnings.

The Origins of Lisp

Lisp, originally spelled LISP, is an acronym for "LISt Processor". The language's creator, John McCarthy, was inspired by the work of his colleagues who in 1956 created IPL (Information Processing Language), an assembly programming language based upon the idea of manipulating lists. Though initially intrigued with their creation, McCarthy's interests in artificial intelligence required a higher-level language than IPL with a more general application of its list-manipulating features. So after much experimentation with FORTRAN, IPL, and heavy inspiration from Alonzo Church's lambda calculus,³ McCarthy created the first version of Lisp in 1958.

In the 1950s and 1960s programming languages were actually created on paper, due to limited computational resources. Volunteers, grad students, and even the children of language creators were used to simulate registers and operations of the language. This is how the first version of Lisp was “run”.⁴ Furthermore, there were two kinds of Lisp: students in the AI lab wrote a form called S-expressions, which was eventually input into an actual computer. These instructions had the form of nested lists, the syntax that eventually became synonymous with Lisp. The other form, called M-expressions, was used when McCarthy gave lectures or presented papers.⁵ These had a syntax which more closely resembles what programmers expect. This separation was natural at the time: for over a decade programmers had entered instructions using binary or machine language while describing these efforts in papers using natural language or pseudocode. McCarthy's students programmed entirely in S-expressions and as their use grew in popularity, the fate of M-expressions was sealed: they were never implemented.⁶

The Lisp 1.5 programmer's manual, first published in 1962, used M-expressions extensively to introduce and explain the language. Here is an example function⁷ for removing spaces from a string input defined using M-expressions:

remsp[string] = [null[string]→F;
              eq[car[string];" "]→member[cdr[string]];
              T→cons[car[string];remsp[cdr[string]]]]

The corresponding S-expression is what the Lisp programmer would actually enter into the IBM 704 machine that was used by the AI lab at MIT:⁸

DEFINE ((
  (REMSP (LAMBDA (STRING)
    (COND ((NULL STRING)
           F)
          ((EQ (CAR STRING) " ")
           (REMSP (CDR STRING)))
          (T
           (CONS (CAR STRING)
                 (REMSP (CDR STRING)))))))))

The period from 1958 to 1962, when Lisp 1.5 was released, marked the beginning of a new era in computer science. Since then Lisp dialects have made an extraordinary impact on the design and theory of other programming languages, changing the face of computing history perhaps more than any other language group. Language features that Lisp pioneered include such significant examples as: homoiconicity, conditional expressions, recursion, meta-programming, meta-circular evaluation, automatic garbage collection, and first class functions. A classic synopsis of these accomplishments was made by the computer scientist of great renown, Edsger Dijkstra in his 1972 Turing Award lecture, where he said the following about Lisp:

“With a few very basic principles at its foundation, [Lisp] has shown a remarkable stability. Besides that, Lisp has been the carrier for a considerable number of, in a sense, our most sophisticated computer applications. Lisp has jokingly been described as ‘the most intelligent way to misuse a computer’. I think that description a great compliment because it transmits the full flavour of liberation: it has assisted a number of our most gifted fellow humans in thinking previously impossible thoughts.”

Lisp usage is generally described as peaking in the 80s and early 90s, experiencing an adoption setback with the widespread view that problems in artificial intelligence were far more difficult to solve than originally anticipated.⁹ Another problem which faced Lisp was related hardware requirements: specialized architectures were developed in order to provide sufficient computational power to its users. These were expensive with few vendors, and a slow product cycle.

In the midst of this Lisp cold-spell, two seminal Lisp books were published: On Lisp, and ANSI Common Lisp, both by famed entrepreneur Paul Graham.¹⁰ Despite a decade of decline, these events helped catalyse a new appreciation for the language by a younger generation of programmers and within a few short years, the number of Lisp books and Lisp-based languages began growing, giving the world the likes of Practical Common Lisp and Let Over Lambda in the case of the former, and Clojure and LFE, in the case of the latter.

Constructing Erlang

Erlang was born in the heart of Ericsson's Computer Science Laboratory,¹¹ just outside of Stockholm, Sweden.¹² The lab had the general aim “to make Ericsson software activities as efficient as possible through the purposeful exploitation of modern technology.” The Erlang programming language was the lab's crowning achievement, but the effort leading up to this was extensive with a great many people engaged in the creation of many prototypes and the use of numerous of programming languages.

One example of this is the work that Nabiel Elshiewy and Robert Virding did in 1986 with Parlog, a concurrent logic programming language based on Prolog. Though this work was eventually abandoned, you can read the paper The Phoning Philosopher's Problem or Logic Programming for Telecommunications Applications and see the impact its features had on the future development of Erlang. The paper provides some examples of Parlog usage; using that for inspiration we can envision what our space-removing program would looking like:¹³

remsp([]) :-
  [].

remsp([$ |Tail]) :-
  remsp(Tail).

remsp([Head|Tail]) :-
  [Head|remsp(Tail)].

Another language that was part of this department-wide experimentation was Smalltalk. Joe Armstrong started experimenting with it in 1985 to model a telephone exchanges and used this to develop a telephony algebra with it. A year later, when his colleague Roger Skagervall showed him the equivalence between this and logic programming, Prolog began its rise to prominence, and the first steps were made towards the syntax of Erlang as the world now knows it. In modern Erlang, our program has the following form:

remsp([]) ->
  [];
remsp([$ |Tail]) ->
  remsp(Tail);
remsp([Head|Tail]) ->
  [Head|remsp(Tail)].

The members of the Ericsson lab who were tasked with building the next generation telephone exchange system, and thus involved with the various language experiments made over the course of a few years, came to the following conclusions:

Small languages seemed better at succinctly addressing the problem space.
The functional programming paradigm was appreciated, if sometimes viewed as awkward.
Logic programming provided the most elegant solutions in the given problem space.
Support for concurrency was viewed as essential.

If these were the initial guideposts for Erlang development, its guiding principles were the following:

To handle high-concurrency
To handle soft real-time constraints
To support non-local, distributed computing
To enable hardware interaction
To support very large scale software systems
To support complex interactions
To provide non-stop operation
To allow for system updates without downtime
To allow engineers to create systems with only seconds of down-time per year
To easily adapt to faults in both hardware and software

These were accomplished using such features as immutable data structures, light weight processes, no shared memory, message passing, supervision trees, and heartbeats. Furthermore, having adopted message-passing as the means of providing high-concurrency, Erlang slowly evolved into the exemplar of a programming paradigm that it essentially invented and even today, dominates: concurrent functional programming.

After four years of development from the late 80s into the early 90s, Erlang matured to the point where it was adopted for large projects inside Ericsson. In 1998 it was released as open source software, and has since seen growing adoption in the wider world of network- and systems-oriented programming.

The Birth of LFE

One of the co-inventors of Erlang, and part of the Lab's early efforts in language experimentation was Robert Virding. Virding first encountered Lisp in 1980 when he started his PhD in theoretical physics at Stockholm University. His exposure to the language came as a result of the physics department's use in performing symbolic algebraic computations. Despite this, he spent more time working on micro-processor programming and didn't dive into it until a few years later when he was working at Ericsson's Computer Science Laboratory. One of the languages evaluated for use in building telephony software was Lisp, but to do so properly required getting to know it in-depth – both a the language level as well as the operating system level.¹⁴ It was in this work that Virding's passion for Lisp blossomed and he came to appreciate deeply its functional nature, macros, and homoiconicity – all excellent and time-saving tools for building complicated systems.

Though the work on Lisps did not become the focus of Erlang development, the seeds of LFE were planted even before Erlang itself had come to be. After 20 years of contributions to the Erlang programming language, these began to bear fruit. In 2007 Virding decided to do something fun in his down time: to see what a Lisp would look like if written on top of the Prolog-inspired Erlang VM. After several months of hacking, he announced a first version of LFE to the Erlang mail list in early 2008.

A few years latter, when asked about the origins of LFE and the motivating elements behind his decision to start the project, Virding shared the following on the LFE mail list:

It had always been a goal of Robert's to make a Lisp which could fully interact with Erlang/OTP, to see what it would look like and how it would run.
He was looking for some interesting programming projects that were not too large to do in his spare time.
He thought it would be a fun, open-ended problem to solve with many interesting parts.
He likes implementing languages.

We showed an example of LFE at the beginning of this chapter; in keeping with our theme for each language subsection, we present it here again, though in a slightly altered form:

(defun remsp
  (('())
   '())
  ((`(32 . ,tail))
   (remsp tail))
  ((`(,head . ,tail))
   (cons head (remsp tail))))

What is LFE Good For?

Very few languages have the powerful capabilities which Erlang offers – both in its standard library as well as the set of Erlang libraries, frameworks, and patterns that are provided in OTP. This covers everything from fault-tolerance, scalability, soft real time capacity, and high-availability to proper design, component assembly, and deployment in distributed environments.

Similarly, despite the impact that Lisp has had on so many programming languages, its full suite of features is still essentially limited to Lisp dialects. This includes the features we have already mentioned: the ability to treat code as data, easily generate new code from data, as well as the interrelated power of writing macros – the last allows developers to modify the language to suit their needs. These rare features from two different language branches are unified in LFE and there is no well-established language that provides the union of these.

As such, LFE gives developers everything they need to envision, prototype, and then build distributed applications – ones with unique requirements that no platform provides and which can be delivered thanks to LFE's language-building capabilities.

To paraphrase and augment the opening of Chapter 1 in Designing for Scalability with Erlang/OTP:

“You need to implement a fault tolerant, scalable soft real time system with requirements for high availability. It has to be event driven and react to external stimulus, load and failure. It must always be responsive. You also need language-level features that don't exist yet. You would like to encode your domain's best practices and design patterns seamlessly into your chosen platform.”

LFE has everything you need to realize this dream ... and so much more.

In Summary

What LFE Is!

Here's what you can expect of LFE:

A proper Lisp-2, based on the features and limitations of the Erlang VM
Compatibility with vanilla Erlang and OTP
It runs on the standard Erlang VM

Furthermore, as a result of Erlang's influence (and LFE's compatibility with it), the following hold:

there is no global data
data is not mutable
only the standard Erlang data types are used
you get pattern matching and guards
you have access to Erlang functions and modules
LFE has a compiler/interpreter
functions with declared arity and fixed number of arguments
Lisp macros

What LFE Isn't

Just to clear the air and set some expectations, we'll go a step further. Here's what you're not going to find in LFE:

An implementation of Scheme
An implementation of Common Lisp
An implementation of Clojure

As such, you will not find the following:

A Scheme-like single namespace
CL packages or munged names faking packages
Access to Java libraries

Notes

¹ Robert Virding, the creator of LFE and one of the co-creators of the Erlang programming language, has previously stated that, were he to start again, he would name his Lisp dialect EFL, since it truly is a Lisp with an Erlang flavour, rather than the other way round.

² We will be covering prefix notation when we cover symbolic expressions later in the book.

³ Alonzo Church was one of McCarthy's professors at Princeton. McCarthy did not use all of the lambda calculus when creating Lisp, as there were many esoteric aspects for which he had no practical need.

⁴ In the case of Lisp, university students were the primary computer hardware ... and sometimes even high school students (see REPL footnote below).

⁵ This approach was not uncommon at the time: the ALGOL 58 specification defined a syntax for the language reference, one for publications, and a third for implementation.

⁶ The single greatest contributor to the ascendance of the S-expression is probably the invention of the REPL by L Peter Deutsch, which allowed for interactive Lisp programming. This was almost trivial in S-expressions, whereas a great deal of effort would have been required to support a similar functionality for M-expressions.

⁷ The function we use in this chapter to demonstrate various syntaxes and dialects was copied from the cover of Byte Magazine's August 1979 issue which focused on Lisp and had part of a Lisp 1.5 program on its cover.

⁸ The formatting applied to the S-expression version of the function is a modern convention, added here for improved readability. There was originally no formatting, since there was no display – a keypunch was used to enter text on punchcards, 80 characters at a time. As such, a more historically accurate representation would perhaps be: DEFINE (((REMSP (LAMBDA (STRING) (COND ((NULL STRING) F) ((EQ (CAR STRING) " ") (REMSP (CDR STRING))) (T (CONS (CAR STRING) (REMSP (CDR STRING)))))))))

⁹ This time period is commonly referred to as the “AI winter”.

¹⁰ Paul Graham sold his Lisp-based e-commerce startup to Yahoo! In 1998.

¹¹ The majority of this section's content was adapted from Joe Armstrong's paper “A History of Erlang” by, written for the HOPL III conference in 2007.

¹² The Computer Science Laboratory operated from 1982 to 2002 in Älvsjö, Stockholm.

¹³ We've taken the liberty of envisioning the Parlog of 1986 as one that supported pattern matching on characters.

¹⁴ One of Virding's project aims was to gain a deeper understanding of Lisp internals. As part of this, he ported the Lisp Machine Lisp object framework Flavors to Portable Standard Lisp running on UNIX. His work on this project contributed to his decision to use Flavour as part of the name for LFE (spelling divergence intentional).

Prerequisites

Anyone coming to LFE should have experience programming in another language, ideally a systems programming language, especially if that language was found lacking. If the corageous reader is attmping to use LFE as a means of entering the study of computer science, we might offer several other paths of study which may bear fruit more quickly and with less pain.

No prior Lisp experience is required, but that would certinaly be helpful. The same goes for Erlang/OTP (or any of the suite of BEAM languages). The reader with experience writing concurrent applications, wrestling with fault-tolerance, or maintaining highly-available applications and services does receive bonus points for preparedness. Such well-prepared readers landing here may have, in fact, done so due to a quest for a distributed Lisp. For those whom this does apply, your quest has found its happy end.

This book assumes the reader has the following installed upon their system:

a package manager for easily installing software (in particular, development tools and supporting libraries)
git, make, and other core open source software development tools
a modern version of Erlang (as of the writing of this book, that would include versions 19 through 23); the rebar3 documentation has great suggestions on what to use here, depending upon your need
the rebar3 build tool for Erlang (and other BEAM languages); see its docs for installation instructions

Conventions

Typography

Key Entry

We use the angle bracket convention to indicate typing actual key on the keyboard. For instance, when the reader sees <ENTER> they should interpret this as an actual key they should type. Note that all keys are given in upper-case. If the reader is expected to use an upper-case "C" instead of a lower-case "c", they will be presented with the key combination <SHIFT><C>.

Code

Color syntax highlighting is used in this text to display blocks of code. The formatting of this display is done in such a way as to invoke in the mind of the reader the feeling of a terminal, thus making an obvious visual distinction in the text. For instance:

(defun fib
  ((0) 0)
  ((1) 1)
  ((n)
    (+ (fib (- n 1))
       (fib (- n 2)))))

Examples such as this one are readily copied and may be pasted without edit into a file or even the LFE REPL itself.

For interactive code, we display the default LFE prompt the reader will see when in the REPL:

lfe> (integer_to_list 42 2)
;; "101010"

We also distinguish the output from the entered LFE code using code comments displayed afer the command.

For shell commands, the commands to enter at the prompt are prefixed by a $ for the prompt. Input and any relevant output are provided as comment strings:

$ echo "I am excited to learn LFE"
# I am excited to learn LFE

LiffyBot

This is a severly hoopy frood. With an attitude. He pops up from time to time, generally with good advice. Or simply as a marker for something the authors hope you will pay special note.

Messages of Note

From time to time you will see call-out boxes, aimed at drawing your attention to something of note. There are four differnt types of these:

ones that share useful info (blue)
ones that highlight something of a momentus nature (green)
ones that offer warnings to tred carefully (orange)
ones that beg you not to follow a particular path (red)

These messages will take the following forms:

Information

Here you will see a message of general interest that could have a useful or even positive impact on your experience in programming LFE.

The icon associated with this type of message is the "i" in a circle.

Amazing!

Here you will see a message of general celebration for sitations that warrant it, above and beyond the general celebration you will feel writing programs in a distributed Lisp.

The icon assocated with this type of message is that of LiffyBot.

Warning!

Here you will see a message indicating a known isssue or practice you should avoid if possible.

The icon assocated with this type of message is the "!" in a caution triangle.

Danger!

Here you will see a message indicating something that could endanger the proper function of an LFE system or threaten the very existence of the universe itself.

The icon assocated with this type of message is "do not enter".

Development Setup

`rebar3` Configuration

Having followed the notes and linked instructions in the Prerequisites section, you are ready to add global support for the LFE rebar3 plugin.

First, unless you have configured other rebar3 plugins on your system, you will need to create the configuration directory and the configuration file:

$ mkdir ~/.config/rebar3
$ touch ~/.config/rebar3/rebar.config

Next, open up that file in your favourite editor, and give it these contents:

{plugins, [
  {rebar3_lfe, "0.4.8"}
]}.

If you already have a rebar.config file with a plugins entry, then simply add a comma after the last plugin listed and paste the {rebar3_lfe, ...} line from above (with no trailing comma!). When a new version of rebar3_lfe is released, you can follow the instructions in the rebar3_lfe repo to upgrade.

For Windows users

Some notes on compatibility:

While LFE, Erlang, and rebar3 work on *NIX, BSD, and Windows systems, much of the development the community does occurs predominently on the first two and sometimes Windows support is spotty and less smooth than on the more used platforms (this is more true for rebar3 and LFE, and _most_ true for LFE).

In particular, starting a REPL in Windows can take a little more effort (an extra step or two) than it does on, for example, Linux and Mac OS X machines.

A Quick Test with the REPL

With the LFE rebar3 plugin successfully configured, you should be able to start up the LFE REPL anywhere on your system with the following:

$ rebar3 lfe repl

Erlang/OTP 23 [erts-11.0] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [hipe]

   ..-~.~_~---..
  (      \\     )    |   A Lisp-2+ on the Erlang VM
  |`-.._/_\\_.-':    |   Type (help) for usage info.
  |         g |_ \   |
  |        n    | |  |   Docs: http://docs.lfe.io/
  |       a    / /   |   Source: http://github.com/rvirding/lfe
   \     l    |_/    |
    \   r     /      |   LFE v1.3-dev (abort with ^G)
     `-E___.-'

lfe>

Exit out of the REPL for now by typing <CTRL><G> and then <Q>.

For Windows users

On Windows, this currently puts you into the Erlang shell, not the LFE REPL. To continue to the LFE REPL, you will need to enter lfe_shell:server(). and then press <ENTER>.

'Hello, World!'

Hello-World style introductory programs are intended to give the prospective programmer for that language a sense of what it is like to write a minimalist piece of software with the language in question. In particular, it should show off the minimum capabilitiues of the language. Practically, this type of program should sigify to the curious coder what they could be in for, should they decide upon this particular path.

In the case of LFE/OTP, a standard Hello-World program (essentially a "print" statement) is extremely misleading; more on that in the OTP version of the Hello-World program. Regardless, we concede to conventional practice and produce a minimal Hello-World that does what many other languages' Hello-World programs do. We do, however, go further afterwards ...

From the REPL

As previously demonstrated, it is possible to start up the LFE 'read-eval-print loop' (REPL) using rebar3:

$ rebar3 lfe repl

Once you are at the LFE prompt, you may write a simple LFE "program" like the following:

lfe> (io:format "~p~n" (list "Hello, World!"))

Or, for the terminally lazy:

lfe> (io:format "~p~n" '("Hello, World!"))

While technically a program, it is not a very interesting one: we didn't create a function of our own, nor did we run it from outside the LFE interactive programming environment.

Let's address one of those points right now. Try this:

lfe> (defun hello-world ()
lfe>   (io:format "~p~n" '("Hello, World!")))

This is a simple function definition in LFE.

We can run it by calling it:

lfe> (hello-world)
;; "Hello, World!"
;; ok

When we execute our hello-world function, it prints our message to standard-output and then lets us know everything really quite fine with a friendly ok.

Note

LFE displays `ok` as output for functions that do not return a value.

Now let's address the other point: running a Hello-World programming from outside LFE.

Hit <CTRL-G><Q> to exit the REPL and get back to your terminal.

From the Command Line

From your system shell prompt, run the following to create a new project that will let us run a Hello-World program from the command line:

$ rebar3 new lfe-main hello-world
$ cd ./hello-world

Once in the project directory, you can actually just do this:

$ rebar3 lfe run

You will see code getting downloaded and compiled, and then your script will run, generating the following output:

Running script '/usr/local/bin/rebar3' with args [] ...
'hello-world

When you created a new LFE project of type 'main', a Hello-World function was automatically generated for you, one that's even simpler than what we created in the previous section:

(defun my-fun ()
  'hello-world)

The other code that was created when we executed rebar3 new lfe-main hello-world was a script meant to be used by LFE with LFE acting as a shell interpreter:

#!/usr/bin/env lfescript

(defun main (args)
  (let ((script-name (escript:script_name)))
    (io:format "Running script '~s' with args ~p ...~n" `(,script-name ,args))
    (io:format "~p~n" `(,(hello-world:my-fun)))))

You may be wondering about the args argument to the main function, and the fact that the printed output for the args when we ran this was []. Let's try something:

$ rebar3 lfe run -- Fenchurch 42

Running script '/usr/local/bin/rebar3' with args [<<"Fenchurch">>,<<"42">>] ...
'hello-world'

We have to provide the two dashes to let rebar3 know that we're done with it, that the subsequent argsuments are not for it, but rather for the program we want it to start for us. Using it causes everything after the -- to be passed as arguments to our script.

As for the code itself, it's tiny. But there is a lot going on just with these two files. Have no fear, though: the remainder of this book will explore all of that and more. For now, know that the main function in the executable is calling the hello-world module's my-fun function, which takes no arguments. To put another way, what we really have here is a tiny, trivial library project with the addition of a script that calls a function from that library.

For now just know that an executable file which starts with #!/usr/bin/env lfescript and contains a main function accepting one argument is an LFE script capable of being executed from the command line -- as we have shown!

LFE/OTP 'Hello, World!'

What have been demonstrated so far are fairly vanilla Hello-World examples; there's nothing particularly interesting about them, which puts them solidly iin the company of the millions of other Hello-World programs. As mentioned before, this approach is particularly vexing in the case of LFE/OTP, since it lures the prospective developer into the preconception that BEAM languages are just like other programming languages. They most decidedly are not.

What makes them, and in this particular case LFE, special is OTP. There's nothing quite like it, certainly not another language with OTP's feature set baked into its heart and soul. Most useful applications you will write in LFE/OTP will be composed of some sort of long-running service or server, something that manages that server and restarts it in the event of errors, and lastly, a context that contains both -- usually referred to as the "application" itself.

As such, a real Hello-World in LFE would be honest and let the prospective developer know what they are in for (and what power will be placed at their fingertips). That is what we will show now, an LFE OTP Hello-World example.

If you are still in the directory of the previous Hello-World project, let's get out of that:

$ cd ../

Now we're going to create a new project, one utilising the some very basic OTP patterns:

$ rebar3 new lfe-app hello-otp-world
$ cd ./hello-otp-world

We won't look at the code for this right now, since there are chapters dedicated to that in the second half of the book. But let's brush the surface with a quick run in the REPL:

$ rebar3 lfe repl

To start your new hello-world application, use the OTP application module:

lfe> (application:ensure_all_started 'hello-otp-world)
;; #(ok (hello-otp-world))

That message lets you know that not only was the hello-otp-word application and server started without issue, any applications upon which it depends were also started. Furthermore, there is a supervisor for our server, and it has started as well. Should our Hello-World server crash for any reason, the supervisor will restart it.

To finish the demonstration, and display the clichéd if classic message:

(hello-otp-world:echo "Hello, OTP World!")
;; "Hello, OTP World!"

And that, dear reader, is a true LFE/OTP Hello-World program, complete with message-passing and pattern-matching!

Feel free to poke around in the code that was generated for you, but know that eventually all its mysteries will be revealed, and by the end of this book this program's magic will just seem like ordinary code to you, ordinary, dependable, fault-tolerant, highly-availble, massively-concurrent code.

Walk-through: An LFE Guessing Game

Now that you've seen some LFE in action, let's do something completely insane: write a whole game before we even know the language!

We will follow the same patterns established in the Hello-World examples, so if you are still in one of the Hello-World projects, change directory and then create a new LFE project:

$ cd ../
$ rebar3 new lfe-app guessing-game
$ cd ./guessing-game

We will create this game by exploring functions in the REPL and then saving the results in a file. Open up your generated project in your favourite code-editing application, and then open up a terminal from your new project directory, and start the REPL:

$ rebar3 lfe repl

Planning the Game

We've created our new project, but before we write even a single atom of code, let's take a moment to think about the problem and come up with a nice solution. Byt doing this, we increase our chances of making something both useful and elegant. As long as what we write remains legible and meets our needs, the less we write the better. This sort of practice elegance will make the code easier to maintain and reduce the chance for bugs (by the simple merrit of there being less code in which a bug may arise; the more code, the greater opportunities for bugs).

Our first step will be making sure we understand the problem and devising some minimal abstractions. Next, we'll think about what actually need to happen in the game. With that in hand, we will know what state we need to track. Then, we're off to the races: all the code will fall right into place and we'll get to play our game.

Key Abstractions

In a guessing game, there are two players: one who has the answer, and one who seeks the answer. Our code and data should clearly model these two players.

Actions

The player with the answer needs to peform the following actions:

At the beginning of the game, state the problem and tell the other player to start guessing
Receive the other player's guess
Check the guess against the answer
Report back to the other player on the provided guess
End the game if the guess was correct

The guessing player needs to take only one action:

guess!

State

We need to track the state of the game. Based upon the actions we've examined, the overall state is very simple. Through the course of the game, will only need to preserve the answer that will be guessed.

Code Explore

Now that we've thought through our problem space clearly and cleanly, let's do some code exploration and start defining some functions we think we'll need.

We've already generated an OTP application using the LFE rebar3 plugin, and once we've got our collection of functions that address the needed game features, we can plug those into the application.

We'll make those changes in the code editor you've opened, and we'll explore a small set of possible functions to use for this using the project REPL session you've just started.

Getting User Input

How do we get user input in LFE? Like this!

lfe> (io:fread "Guess number: " "~d")

This will print the prompt Guess number: and then await your input and the press of the <ENTER> key. The input you provide needs to match the format type given in the second argument. In this case, the ~d tells us that this needs to be a decimal (base 10) integer.

fe> (io:fread "Guess number: " "~d")
;; Guess number: 42
;; #(ok "*")

If we try typing something that is not a base 10 integer, we get an error:

lfe> (io:fread "Guess number: " "~d")
;; Guess number: forty-two
;; #(error #(fread integer))

With correct usage, how do we capture the value in a variable? The standard way to do this in LFE is destructuring via pattern matching. The following snippet extracts the value and then prints the extracted value in the REPL:

lfe> (let ((`#(ok (,value)) (io:fread "Guess number: " "~d")))
lfe>   (io:format "Got: ~p~n" `(,value)))
;; Guess number: 42
;; Got: 42
;; ok

We'll talk a lot more about pattern matching in the future, as well as the meaning of the backtick and commas. For now,let's keep pottering in the REPL with these explorations, and make a function for this:

lfe> (defun guess ()
lfe>   (let ((`#(ok (,value)) (io:fread "Guess number: " "~d")))
lfe>     (io:format "You guessed: ~p~n" `(,value))))

And call it:

lfe> (guess)
;; Guess number: 42
;; You guessed: 42
;; ok

Checking the Input

In LFE there are several ways in which you can perform checks on values:

the if form
the cond form
the case form
pattern-matching and/or guards in function heads

The last one is commonly used in LFE when passing messages / data between functions. Our initial, generated project code is already doing this, and given the game state data we will be working with, this feels like a good fit what we need to implement.

Normally records are used for application data, but since we just care about the value of two integers (the number selected for the answer and the number guessed by the player), we'll keep things simple in this game:

(set answer 42)

Let's create a function with a guard:

lfe> (defun check
lfe>   ((guess) (when (< guess answer))
lfe>    (io:format "Guess is too low~n")))

The extra parenthesis around the function's arguments is due to the use of the pattern-matching form of function definition we're using here. We need this form, since we're going to use a guard. The when after the function args is called a "guard" in LFE. As you might imagine, we could use any number of these.

lfe> (check 10)
;; Guess is too low
;; ok

Let's add some more guards for the other checks we want to perform:

lfe> (defun check
lfe>   ((guess) (when (< guess answer))
lfe>    (io:format "Guess is too low~n"))
lfe>   ((guess) (when (> guess answer))
lfe>    (io:format "Guess is too high~n"))
lfe>   ((guess) (when (== guess answer))
lfe>    (io:format "Correct!~n")))

lfe> (check 10)
;; Guess is too low
;; ok

lfe> (check 100)
;; Guess is too high
;; ok

lfe> (check 42)
;; Correct!
;; ok

This should give a very general sense of what is possible.

Integrating into an Application

We're only going to touch one of the files that was generated when you created the guessing-game project: ./src/guessing-game.lfe. You can ignore all the others. Once we've made all the changes summarized below, we will walk through this file at a high level, discussing the changes and how those contribute to the completion of the game.

First though, we need to reflect on the planning we just did, remembering the actions and states that we want to support. There's also another thing to consider, since we're writing this as is an always-up OTP app. With some adjustments for state magagement, it could easily be turned into something that literally millions of users could be accessing simultaneouslyi. So: how does a game that is usually implemented as a quick CLI toy get transformed in LFE/OTP such that it can be run as a server?

In short, we'll use OTP's gen_server capability ("behaviour") and the usual message-passing practices. As such, the server will need to be able to process the following messages:

#(start-game true) (create a record to track game state)
#(stop-game true) (clear the game state)
#(guess n)
- check for guess equal to the answer
- greater than the answer, and
- less than the answer

We could have just used atoms for the first two, and done away with the complexity of using tuples for those, but symmetry is nice :-)

To create the game, we're going to need to perform the following integration tasks:

Update the handle_cast function to process the commands and guards we listed above
Create API functions that cast the appropriate messages
Update the export form in the module definition
Set the random seed so that the answers are different every time you start the application

`handle_cast`

The biggest chunk of code that needs to be changed is the handle_cast function. Since our game doesn't return values, we'll be using handle_cast. (If we needed to have data or results returned to us in the REPL, we would have used handle_call instead. Note that both are standard OTP gen_server callback functions.)

The generated project barely populates this function and the function isn't of the form that supports patten-matching (which we need here) so we will essentially be replacing what was generated. In the file ./src/guessing-game.lfe, change this:

(defun handle_cast (_msg state)
  `#(noreply ,state))

to this:

(defun handle_cast
  ((`#(start-game true) _state)
   (io:format "Guess the number I have chosen, between 1 and 10.~n")
   `#(noreply ,(random:uniform 10)))
  ((`#(stop-game true) _state)
   (io:format "Game over~n")
   '#(noreply undefined))
  ((`#(guess ,n) answer) (when (== n answer))
   (io:format "Well-guessed!!~n")
   (stop-game)
   '#(noreply undefined))
  ((`#(guess ,n) answer) (when (> n answer))
   (io:format "Your guess is too high.~n")
   `#(noreply ,answer))
  ((`#(guess ,n) answer) (when (< n answer))
   (io:format "Your guess is too low.~n")
   `#(noreply ,answer))
  ((_msg state)
   `#(noreply ,state)))

That is a single function in LFE, since for every match the arity of the function remains the same. It is, however, a function with six different and separate arguement-body forms: one for each pattern and/or guard.

These patterns are matched:

start
stop
guess (three times)
any

For the three guess patterns (well, one pattern, really) since there are three different guards we want placed on them:

guess is equal
guess is greater
guess is less

Note that the pattern for the function argument in these last three didn't change, only the guard is different beptween them.

Finally, there's the original "pass-through" or "match-any" pattern (this is used to prevent an error in the event of an unexpected message type).

Game API

In order to send a message to a running OTP server, you use special OTP functions for the type of server you are running. Our game is running a gen_server so we'll be using that OTP module to send messages, in particular we'll be calling gen_server:cast. However, creating messages and sending them via the appropriate gen_server function can get tedious quickly, so it is common practice to create API functions that do these things for you.

In our case, we want to go to the section with the heading ;;; our server API and add the following:

(defun start-game ()
  (gen_server:cast (SERVER) '#(start-game true)))

(defun stop-game ()
  (gen_server:cast (SERVER) '#(stop-game true)))

(defun guess (n)
  (gen_server:cast (SERVER) `#(guess ,n)))

Functions in LFE are private by default, so simply adding these functions doesn't make them publicly accessible. As things now stand these will not be usable outside their module; if we want to use them, e.g., from the REPL, we need to export them.

Go to the top of the guessing-game module and update the "server API" sectopm of the exports, chaning this:

    ;; server API
    (pid 0)
    (echo 1)))

to this:

    ;; server API
    (pid 0)
    (echo 1)
    (start-game 0)
    (stop-game 0)
    (guess 1)))

The final form of your module definition should look like this:

(defmodule guessing-game
  (behaviour gen_server)
  (export
    ;; gen_server implementation
    (start_link 0)
    (stop 0)
    ;; callback implementation
    (init 1)
    (handle_call 3)
    (handle_cast 2)
    (handle_info 2)
    (terminate 2)
    (code_change 3)
    ;; server API
    (pid 0)
    (echo 1)
    (start-game 0)
    (stop-game 0)
    (guess 1)))

Now our game functions are public, and we'll be able to use them from the REPL.

Finishing Touches

There is one last thing we can do to make our game more interesting. Right now, the game will work. But every time we start up the REPL and kick off a new game, the same "random" number will be selected for the answer. In order to make things interesting, we need to generate a random seed when we initialize our server.

We want to only do this once, though -- not every time the game starts, and certainly not every time a user guesses! When the LFE server supervisor starts our game server, one functions is called and called only once: init/1. That's where we want to make the change to support a better-than-default random seed.

Let's change that function:

(defun init (state)
  `#(ok ,state))

to this:

(defun init (state)
  (random:seed (erlang:phash2 `(,(node)))
               (erlang:monotonic_time)
               (erlang:unique_integer))
  `#(ok ,state))

Now we're ready to play!

Playing the Game

If you are still in the REPL, quit out of it so that rebar3 can rebuild our changed module. Then start it up again:

$ rebar3 lfe repl

Once at the LFE propmpt, start up the application:

lfe> (application:ensure_all_started 'guessing-game)

With the application and all of its dependencies started, we're ready to start the game and play it through:

lfe> (guessing-game:start-game)
;; ok
;; Guess the number I have chosen, between 1 and 10.

lfe> (guessing-game:guess 10)
;; ok
;; Your guess is too high.

lfe> (guessing-game:guess 1)
;; ok
;; Your guess is too low.

lfe> (guessing-game:guess 5)
;; ok
;; Your guess is too low.

lfe> (guessing-game:guess 7)
;; ok
;; Your guess is too low.

lfe> (guessing-game:guess 8)
;; ok
;; Well-guessed!!
;; Game over

Success! You've just done something pretty amazing, if still mysterious: you've not only created your first OTP application running a generic server, you've successully run it through to completion!

Until we can dive into all the details of what you've seen in this walkthrough, much of what you've just written will seem strange and maybe even overkill. For now, though, we'll mark a placeholder for those concepts: the next section will briefly review what you've done and indicate which parts of this book will provide the remaining missing pieces.

Review

We've got the whole rest of the book ahead of us to cover much of what you've seen in the sample project we've just created with our guessing game. In the coming pages, you will revisit every aspect of what you've seen so far in lots of detail with correspondingly useful insttruction on these matters.

That being said, it would be unfair to not at least read through the code together and mention the high-level concepts involved. Since we only touched the code in one file, that will be the one that gets the most of our attention for this short review, but let's touch on the others here, too.

Project Files

`rebar.config`

This is the file you need in every LFE project you will write in order to take advantage of the features (and time-savings!) that rebar3 provides. For this project, the two important parts are:

the entry for dependencies (only LFE in this case), and
the plugins entry for the LFE rebar3 plugin.

Project setup will be covered in Chapter XXX, section XXX.

Source Files

The source files for our sample program in this walkthrough are for an OTP application. OTP-based projects will be covered in Chapter XXX, section XXX.

`.app.src`

This file is mostly used for application metadata. Most of what our app uses in this file is pretty self-explanatory. Every LFE application will have one of these in the project source code. Every LFE library and application needs this file.

`guessing-game-app.lfe`

This is the top-level file for our game, an OTP application. It only exports two functions: one to start the app and the other to stop it. The application is responsible for starting up whatever supervisors all your services/servers need. For this sample application, only one supervisor is needed (with a very simple supervision tree).

`guessing-game-sup.lfe`

This module is a little more invloved and has all the configuration and code necessary to properly set up a supervisor for our server. When something goes wrong with our server, the restart strategy defined by our supervisor will kick in and get things back up and running again. This is one of the key secrets to OTP's wizardry, and we will be covering this in great detail later.

`src/guessing-game.lfe`

This is the last file we'll look at, and is the one we'll cover in the most detail right now. Here's the entire content of what we created for our game:

(defmodule guessing-game
  (behaviour gen_server)
  (export
   ;; gen_server implementation
   (start_link 0)
   (stop 0)
   ;; callback implementation
   (init 1)
   (handle_call 3)
   (handle_cast 2)
   (handle_info 2)
   (terminate 2)
   (code_change 3)
   ;; server API
   (pid 0)
   (echo 1)
   (start-game 0)
   (stop-game 0)
   (guess 1)))

;;; ----------------
;;; config functions
;;; ----------------

(defun SERVER () (MODULE))
(defun initial-state () '#())
(defun genserver-opts () '())
(defun unknown-command () #(error "Unknown command."))

;;; -------------------------
;;; gen_server implementation
;;; -------------------------

(defun start_link ()
  (gen_server:start_link `#(local ,(SERVER))
                         (MODULE)
                         (initial-state)
                         (genserver-opts)))

(defun stop ()
  (gen_server:call (SERVER) 'stop))

;;; -----------------------
;;; callback implementation
;;; -----------------------

(defun init (state)
  (random:seed (erlang:phash2 `(,(node)))
               (erlang:monotonic_time)
               (erlang:unique_integer))
  `#(ok ,state))

(defun handle_cast
  ((`#(start-game true) _state)
   (io:format "Guess the number I have chosen, between 1 and 10.~n")
   `#(noreply ,(random:uniform 10)))
  ((`#(stop-game true) _state)
   (io:format "Game over~n")
   '#(noreply undefined))
  ((`#(guess ,n) answer) (when (== n answer))
   (io:format "Well-guessed!!~n")
   (stop-game)
   '#(noreply undefined))
  ((`#(guess ,n) answer) (when (> n answer))
   (io:format "Your guess is too high.~n")
   `#(noreply ,answer))
  ((`#(guess ,n) answer) (when (< n answer))
   (io:format "Your guess is too low.~n")
   `#(noreply ,answer))
  ((_msg state)
   `#(noreply ,state)))

(defun handle_call
  (('stop _from state)
   `#(stop shutdown ok state))
  ((`#(echo ,msg) _from state)
   `#(reply ,msg state))
  ((message _from state)
   `#(reply ,(unknown-command) ,state)))

(defun handle_info
  ((`#(EXIT ,_from normal) state)
   `#(noreply ,state))
  ((`#(EXIT ,pid ,reason) state)
   (io:format "Process ~p exited! (Reason: ~p)~n" `(,pid ,reason))
   `#(noreply ,state))
  ((_msg state)
   `#(noreply ,state)))

(defun terminate (_reason _state)
  'ok)

(defun code_change (_old-version state _extra)
  `#(ok ,state))

;;; --------------
;;; our server API
;;; --------------

(defun pid ()
  (erlang:whereis (SERVER)))

(defun echo (msg)
  (gen_server:call (SERVER) `#(echo ,msg)))

(defun start-game ()
  (gen_server:cast (SERVER) '#(start-game true)))

(defun stop-game ()
  (gen_server:cast (SERVER) '#(stop-game true)))

(defun guess (n)
  (gen_server:cast (SERVER) `#(guess ,n)))

The beginning of the file opens with a declaration of the module: not only its name, but the public functions we want to expose as part of our API. This will be covered in Chapter XXX, section XXX.

Next, we have a few constant functions. Functions are necessary here due to the fact that LFE does not have global variables. This will be covered in Chapter XXX, section XXX.

Then we define the functions that will be used as this module's implementation of a generic OTP server. There is some boilerplate here that will be discussed when we dive into LFE/OTP. This will be covered in Chapter XXX, section XXX.

After that, we define the functions that are used by the OTP machinery that will run our server. Here you see several examples of pattern matching function heads in LFE, a very powerful feature that lends itself nicely to consise and expressive code. This will be covered in Chapter XXX, section XXX.

Lastly, we define our own API. Most of these functions simply send messages to our running server. More on this in Chapter XXX, section XXX.

The LFE REPL

We briefly introduced the REPL in the first version of the Hello-World example we wrote, stating that it was an acronym for 'read-eval-print loop' and how to start it with rebar3. As an LFE developer, this is one of the primnary tools -- arguably the most powerful -- at your disposal, so we're going to do a more thorough job of introducing its capabilities in this section.

Historical Note

The first Lisp interpreter was created sometime in late 1958 by then-grad student Steve Russell after reading John McCarthy's definition of eval. He had the idea that the theoretical description provided there could actually be implemented in machine code.

In 1963 L Peter Deutsch, a high school student at the time, combined the read, eval, and print core functions to create the first REPL (or, as he termed it then, the 'READ-EVAL-PRINT cycle'). This was done as part of his successful effort to port Lisp 1.5 from the IBM 7090 to the DEC PDP-1 and is referenced briefly in a written report filed with the Digital Equipment Computer Users Society in 1964.

A basic REPL can be implemented with just four functions; such an implementation could be started with the following:

(LOOP (PRINT (EVAL (READ))))

LFE has implemented most these functions for us already (and quite robustly), but we could create our own very limited REPL (single lines with no execution context or environment) within the LFE REPL using the following convenience wrappers:

(defun read ()
  (case (io:get_line "myrepl> ")
    ("quit\n" "quit")
    (str (let ((`#(ok ,expr) (lfe_io:read_string str)))
           expr))))

(defun print (result)
  (lfe_io:format "~p~n" `(,result))
  result)

(defun loop
  (("quit")
   'good-bye)
  ((code)
   (loop (print (eval (read))))))

Now we can start our custom REPL inside the LFE REPL:

lfe> (loop (print (eval (read))))

This gives us a new prompt:

myrepl>

At this prompt we can evaluate basic LFE expressions:

myrepl> (+ 1 2 3)
;; 6
myrepl> (* 2 (lists:foldl #'+/2 0 '(1 2 3 4 5 6)))
;; 42
myrepl> quit
;; good-bye
lfe>

Note that writing an evaluator is the hard part, and we've simply re-used the LFE evaluator for this demonstration.

Now that we've explored some of the background of REPLs and Lisp interpreters, let's look more deeply into the LFE REPL and how to best take advantage of its power when using the machine that is LFE and OTP.

Core Features

In keeping with the overall herritage of LFE, its REPL is both a Lisp REPL as well as an Erlang shell. In fact, when support was added for the LFE REPL to the rebar3_lfe plugin, it utilised all of the plumbing for the Erlang shell support in rebar3.

For the developer, though, this means that the LFE REPL holds a dual set of features, multiplying the set of features available to just the Erlang shell. These features include the following support:

Evaluation of Lisp S-expressions
Definition of functions, completely with LFE tail-recursion support
Definition of records and use of record-specific support functions/macros
Creation of LFE macros via standard Lisp-2 syntax
Macro examination and debugging with various expansion macros
The ability to start the LFE REPL in distribution mode, complete with Erlang cookie, and thus to not only access remote LFE and Erlang nodes, but to be accessed as a remote node itself (for nodes that have been granted access
Access to the Erlang JCL and the ability to start separate, LFE shells running concurrently

Unsupported

The following capabilities are not supported in the LFE REPL:

Module definitions; these are a file-based feature in LFE, just as with Erlang.

Starting LFE

The `lfe` executable

While this book focuses upon the use of rebar3 and its LFE plugin -- due entirely to the amount of time it saves through various features it supports -- LFE may be used quite easily without it.

To use LFE and its REPL without rebar3, you'll need to clone the repo, e.g.:

$ cd ~/lab
$ git clone https://github.com/rvirding/lfe.git
$ cd lfe

Since you have read the earlier section on dependencies, you already have Erlang, make, and your system build tools installed. As such, all you have to do is run the following to build LFE:

$ make

This will generate an executable in ./bin and you can start the LFE REPL by calling it:

$ ./bin/lfe

Erlang/OTP 23 [erts-11.0.2] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [hipe] [dtrace]

   ..-~.~_~---..
  (      \\     )    |   A Lisp-2+ on the Erlang VM
  |`-.._/_\\_.-':    |   Type (help) for usage info.
  |         g |_ \   |
  |        n    | |  |   Docs: http://docs.lfe.io/
  |       a    / /   |   Source: http://github.com/rvirding/lfe
   \     l    |_/    |
    \   r     /      |   LFE v1.3-dev (abort with ^G)
     `-E___.-'

lfe>

If you opt to install LFE system-wide with make install, then you can start the REPL from anywhere by simply executing lfe.

Via `rebar3 lfe repl`

As demonstrated earlier on several occasions, you can start the LFE REPL with the rebar3 LFE plugin (and this is what we'll do in the rest of this manual):

$ rebar3 lfe repl

Since you have updated your global rebar3 settings (in the "Prerequisites" section, after following the instructions on the rebar3 site), you may also start the LFE REPL from anywhere on your machine using the rebar3 command.

`readline` Support

History

Tab-Completion

`(help)`

As you gain familiarity with the LFE REPL, one of the most useful and convenient references will be the summary of functions, commands, and variablese that come with the LFE REPL.

To see these, simple call the help or h function:

lfe> (help)

That will result in the following being displyed to your terminal:

LFE shell built-in functions

(c file)       -- compile and load code in <file>
(cd dir)       -- change working directory to <dir>
(clear)        -- clear the REPL output
(doc mod)      -- documentation of a module
(doc mod:mac)  -- documentation of a macro
(doc m:f/a)    -- documentation of a function
(ec file)      -- compile and load code in erlang <file>
(ep expr)      -- print a term in erlang form
(epp expr)     -- pretty print a term in erlang form
(exit)         -- quit - an alias for (q)
(flush)        -- flush any messages sent to the shell
(h)            -- an alias for (help)
(help)         -- help info
(i)            -- information about the system
(i pids)       -- information about a list of pids
(l module)     -- load or reload <module>
(ls)           -- list files in the current directory
(ls dir)       -- list files in directory <dir>
(m)            -- which modules are loaded
(m mod)        -- information about module <mod>
(p expr)       -- print a term
(pp expr)      -- pretty print a term
(pid x y z)    -- convert <x>, <y> and <z> to a pid
(pwd)          -- print working directory
(q)            -- quit - shorthand for init:stop/0
(regs)         -- information about registered processes

LFE shell built-in commands

(reset-environment)             -- reset the environment to its initial state
(run file)                      -- execute all the shell commands in a <file>
(set pattern expr)
(set pattern (when guard) expr) -- evaluate <expr> and match the result with
                                   pattern binding
(slurp file)                    -- slurp in a LFE source <file> and makes
                                   everything available in the shell
(unslurp)                       -- revert back to the state before the last
                                   slurp

LFE shell built-in variables

+/++/+++      -- the tree previous expressions
*/**/***      -- the values of the previous expressions
-             -- the current expression output
$ENV          -- the current LFE environment

ok

Most of those are documented in stdlib reference for their Erlang counterparts, so be sure to reference that information for details on many of the above.

Those not covered in that Erlang reference manual, or those that are different in their LFE versionsm, include:

Built-in Functions
- Compilation functions
- LFE code documentation
- Printing and pretty-printing
Built-in commands
Built-in variables

REPL Functions

Most of the LFE REPL functions are documented in stdlib reference for their Erlang counterparts. This section documents where the LFE REPL help diverges from the Erlang Shell help.

Compilation

If you view the Erlang reference manual documentation for compiling files in the shell, you will see differences from what is show in the LFE help text. In particular, (c) is for compiling LFE modules and (ec) needs to be used for compiling Erlang source files.

In both cases, the resulting .beam files are compiled to the current working directory and not to an ebin directory. These .beam files will be found by LFE, since the current working directory is included in the path, but you'll likely want to perform some cleanup afterward.

Documentation

You may access the documentation for LFE modules, macros, and functions in the REPL via the doc function. For instance, the Common Lisp compatibility module's documentation:

lfe> (doc cl)
;; ____________________________________________________________
;; cl
;;
;; LFE Common Lisp interface library.
;;
;; ok

That module's cond macro documentation:

lfe> (doc cl:cond)
;; ____________________________________________________________
;; cond
;; args
;; CL compatible cond macro.
;;
;; ok

That module's pairlis/2 function documentation:

lfe> (doc cl:pairlis/2)
;; ____________________________________________________________
;; pairlis/2
;; keys values
;; Make an alist from pairs of keys values.
;;
;; ok

Documentation for Erlang modules and fucntions is available via the Command Interface

Printing Data

LFE Formatting

LFE provides some nice convenience functions for displaying data structions in the REPL. Let's say we had a data structure defined thusly:

lfe> (set data `(#(foo bar baz) #(quux quuz) #(corge grault garply)
lfe> #(plugh xyzzy) #(flurb nirf) #(zobod zirfid)))

We can print our data with the following:

lfe> (p data)
;; (#(foo bar baz) #(quux quuz) #(corge grault garply) #(plugh xyzzy) #(flurb nirf) #(zobod zirfid))
;; ok

Or we can pretty-print it:

lfe> (pp data)
;;(#(foo bar baz)
;; #(quux quuz)
;; #(corge grault garply)
;; #(plugh xyzzy)
;; #(flurb nirf)
;; #(zobod zirfid))
;; ok

Erlang Formatting

The same may be done for displaying data in the Erlang format:

lfe> (ep data)
;; [{foo,bar,baz},{quux,quuz},{corge,grault,garply},{plugh,xyzzy},{flurb,nirf},{zobod,zirfid}]
;; ok

lfe> (epp data)
;; [{foo,bar,baz},
;;  {quux,quuz},
;;  {corge,grault,garply},
;;  {plugh,xyzzy},
;;  {flurb,nirf},
;;  {zobod,zirfid}]
;; ok

REPL Commands

The LFE REPL provides several useful commands users:

(reset-environment)             -- reset the environment to its initial state
(run file)                      -- execute all the shell commands in a <file>
(set pattern expr)
(set pattern (when guard) expr) -- evaluate <expr> and match the result with
                                   pattern binding
(slurp file)                    -- slurp in a LFE source <file> and makes
                                   everything available in the shell
(unslurp)                       -- revert back to the state before the last
                                   slurp

These are fairly self-explanatory, with the possible exception of clarifying how run and slurp differ:

Calling (run "some/file.lfe") will cause the LFE REPL to read the contents of that file and then execute every line in that file as if they had been typed at the terminal. This is a convenient way of duplicating REPL state between sessions. (If you haven't kept track of your entries, you can always open up your BEAM history file and create an .lfe file with all the required commands!)
Calling (slurp "some/other/file.lfe") will place all functions, records, and macros defined in that file into the LFE environment, allowing you to call them without a module: prefix. Note that the code is not executed, but is instead placed into the current environment, ready for use.

Special Variables

LFE shell built-in variables

+/++/+++      -- the tree previous expressions
*/**/***      -- the values of the previous expressions
-             -- the current expression output
$ENV          -- the current LFE environment

Most of these variables are taken directly from Common Lisp and have the same exact meaning. From the Common Lisp HyperSpec for +,++,+++:

The variables +, ++, and +++ are maintained by the Lisp read-eval-print loop to save forms that were recently evaluated.

The value of + is the last form that was evaluated, the value of ++ is the previous value of +, and the value of +++ is the previous value of ++.

And for *,**,***:

The variables *, **, and *** are maintained by the Lisp read-eval-print loop to save the values of results that are printed each time through the loop.

The value of * is the most recent primary value that was printed, the value of ** is the previous value of *, and the value of *** is the previous value of **.

Lastly, for -:

The value of - is the form that is currently being evaluated by the Lisp read-eval-print loop.

The $ENV variable in the LFE REPL is a critical tool for debugging particularly tricky issues in the REPL (especially useful when creating complex macros).

Command Interface

While many of the functions listed in the LFE (help) have their documentation in the Erlang Command Interface module (CIM), not everything in the CIM has been provided in the LFE REPL, some of which can be useful at times.

Here are some of the more useful functions you may with to be aware of from that module:

(c:bt pid) - Stack backtrace for a process. Equivalent to (erlang:process_display pid 'backtrace).
(c:h mod) - Print the documentation for mod
(c:h mod fun) - Print the documentation for all mod:fun
(c:h mod fun arity) - Print the documentation for mod:fun/arity
(c:lm) - Reloads all currently loaded modules that have changed on disk (see (c:mm)). Returns the list of results from calling (l mod) for each such loaded module.
(c:memory) - Memory allocation information. Equivalent to (erlang:memory).
(c:mm) -
(c:ni) - Display system information, listing information about all nodes on the network
(c:nl mod) - Loads Module on all nodes
(c:nregs) - Displays information about all registered processes for all nodes in the network.
(c:uptime) - Prints the node uptime (as specified by (erlang:statistics 'wall_clock)) in human-readable form.
(c:xm mod) - Finds undefined functions, unused functions, and calls to deprecated functions in a module by calling xref:m/1.

Job Control

When in the LFE REPL, a special mode is accessible upon typing <CTRL-G>:

lfe>
User switch command
 -->

At the JCL --> prompt, you may get help text by typing ?:

 --> ?
  c [nn]            - connect to job
  i [nn]            - interrupt job
  k [nn]            - kill job
  j                 - list all jobs
  s [shell]         - start local shell
  r [node [shell]]  - start remote shell
  q                 - quit erlang
  ? | h             - this message
 -->

Running Multiple Shells

[ also: custom prompts ]

Files

REPL-Only

`slurp`

`unslurp`

Compiling LFE Files

Compiling Erlang Files

REPL & Module

`include-lib`

`include-file`

Module Loading

l and code:ensure_loaded

Part II - Code as Data

Variables

In LFE, variables are implemented with atoms. Atoms are used in the language for such things as naming functions and macros, use as keywords and in data structures. As you'll find out when reading about atoms, they are evaluated as simply themselves. However, when they are used as variable names, they evaulate to the value of which they were assigned.

There are two contexts for setting variables:

in the LFE REPL
in LFE modules

This distinction is important, not only because the contexts use different forms, but because like Erlang, LFE does not support global variables.

This chpater will also introduce the reader to pattern-matching in LFE, as it applies to variable assignment; a fuller discussion is presented in a dedicated chapter as well in the chapters that cover forms which support pattern-matching (i.e., compound data types, functions, etc.).

Lastly we'll talk more about some of the LFE nuances around global variables.

Bindings

In the REPL

To set a variable in the LFE REPL, use the set macro:

lfe> (set answer 42)
42

In the language itself, LFE doesn't support global variables -- a valuable feature inherited from Erlang. However, in order for a REPL experience to be useful, an environment must be maintained in which the user may write expressions and then refer to them later. This environment is essentially a mechanism for global state in the context of a single user running a single Erlang VM. If we set a variable called answer, that variable will be available to us as long as the REPL process continues or until we reset the environment.

Setting another value with the same variable name is allowed: it merely replaces the assignment in the current REPL environment:

lfe> (set answer "forty-two")
"forty-two"

With a variable assigned with set it may be used at any time in the REPL environment where it was defined:

lfe> (++ "The answer is " answer)
"The answer is forty-two"

Attempting to use a variable that has not been defined results in an error:

lfe> (++ "The question was " question)
** exception error: symbol question is unbound
  in lfe_eval:eval_error/1 (src/lfe_eval.erl, line 1292)
  in lists:map/2 (lists.erl, line 1243)
  in lists:map/2 (lists.erl, line 1243)

If you don't have any need to update the environment with data that you only need for a specific calculation, you may use the let form:

lfe> (let ((short-lived-value (* 2 (+ 1 2 3 4 5 6))))
lfe>   (io:format "The answer is ~p~n" `(,short-lived-value)))
The answer is 42
ok

Let's make sure that variable wasn't saved to our environment:

lfe> short-lived-value
** exception error: symbol short-lived-value is unbound

The lexical scope for the short-lived-value is within the let only and is not available outside of that scope.

In Functions and Macros

Within functions, variables are lexically scoped and bound with let and let*. One may also define lexically scoped functions inside other fucntions, and this is done with flet and fletrec (the latter required for defining recursive functions inside another function). These will be covered in detail later in the book.

We've seen let used above in the REPL; the same applies inside functions:

(defun display-answer ()
  (let ((answer (* 2 (+ 1 2 3 4 5 6))))
    (io:format "The answer is ~p~n" `(,answer))))

This is a straight-forward case of assignment; but what if we needed to assign a varaible that depended upon another variable. Using let, you'd have to do this:

(defun display-answer ()
  (let ((data '(1 2 3 4 5 6)))
    (let ((answer (* 2 (lists:sum data))))
      (io:format "The answer is ~p~n" `(,answer)))))

However, as with other Lisps, LFE provides a convenience macro for this: let*. Here's how it's used:

(defun display-answer ()
  (let* ((data '(1 2 3 4 5 6))
         (answer (* 2 (lists:sum data))))
    (io:format "The answer is ~p~n" `(,answer))))

Lexical scoping helps one isolate unrelated data or calculations, even in the same function: multiple let or let* blocks may be declared in a function and none of the bound variables in one block will be available to another block. Attempting to do so will result in an error.

In Modules

In LFE, one cannot bind variables at the module-level, only functions and macros. This is part of the "no global variables" philosophy (and practice!) of Erlang and LFE. Module-level bindings are done with defun for functions and defmacro for macros. The creation of modules, functions, and macros will all be covered in detail later in the book.

Shadowing

One shadows a variable in one scope when, at a higher scope, that variable was also defined. Here's an annotated example:

(defun shadow-demo ()
  (let ((a 5))
    (io:format "~p~n" `(,a))    ; prints 5
    (let ((a 'foo))             ; 'a' here shadows 'a' in the previous scope
      (io:format "~p~n" `(,a))) ; prints foo
    (io:format "~p~n" `(,a)))   ; prints 5; the shadow binding is out of scope
  (let ((a 42))
    (io:format "~p~n" `(,a))))  ; prints 42 - new scope, no shadowing

Shadowing also may occur at the module-level with the definition of functions, and the shadowing could be of functions at one of several levels. Here's a run-down on function shadowing in modules, from the highest or "outermost" to the lowest or "innermost":

Predefined Erlang built-in functions (BIFs) may be shadowed by any of the following
Predefined LFE BIFs may be shadowed by any of the following
Module imports may shadow any of the above via aliasing
Functions defined in a module may shadow any of the above
Functions defined inside a function (e.g., via flet or fletrec) may shadow any of the above

Note that to shadow functions in LFE, functions must match both in name as well as arity (number of arguments).

The hd Erlang BIF returns the "head" of a list (the first item). Here's an example of shadowing it in the REPL. Here's the BIF at work:

lfe> (hd '(a b c d e))
a

Next, paste this into the REPL:

(defun hd (_)
  ;; part of the pun here is that the same function in Lisp is called 'car'
  "My other car is The Heart of Gold.")

The hd function takes one argument (a list), so our function also needs to take one. However, since we don't do anything with that, we use the "don't care" variable _.

Now let's call hd again:

lfe> (hd '(a b c d e))
"My other car is The Heart of Gold."

Shadowed!

Note that, like many other Lisps, LFE has the car function, but since this is a core form, it can't be shadowed (see the next section).

The Unshadowable

Shadowing does not apply to the supported LFE core forms. It may appear that your code is shadowing those forms, but the compiler will always use the core meaning and never an alternative. It does this silently, without warning -- so take care and do not be surprised!

Information

Core LFE forms are never be shadowed.

Pattern-matching Preview

Global Variables Revisted

The Process Dictionary

ETS Tables

State and OTP Servers

External Databases

Primitive Types

This chapter covers the basic types of data available to LFE, upon which primatives rest all LFE libraries and applications.

Integers
Floats
Atoms
Booleans
Characters

Each of these types has an LFE test function of the form TYPENAMEp (which wrap the respective Erlang is_TYPENAME function). These are used to perform type checks (especially common in guard expressions). These predicate functions will be covered in their respective type sections.

Integers

Integers in LFE may be either positive or negative and are by default base 10. Like Erlang, LFE does not have a maximum size of integer (in contrast to languages like C). This is accomplished via automatic conversion to larger (or smaller) internal representations (including the use of bignums).

lfe> 42
42
lfe> -42
-42
lfe> 1764
1764
lfe> 150130937545296561928688012959677941476701514734130607701636390322176
150130937545296561928688012959677941476701514734130607701636390322176

Bases

Several bases are supported via special syntax:

lfe> #b101010 ; binary
42
lfe> #o52 ; octal
42
lfe> #d42 ; decimal (explicit base 10)
42)
lfe> #x2a ; hexadecimal
42

Generic bases are supported, too:

lfe> #36r16
42

The number after the hash # is the base and may be any positive integer from 2 through 36. The number after the radix r is the actual value and must only bve comprised of integers allowed for the given base.

Converting between bases in LFE is most easily done via the integer_to_list Erlang function. For example:

lfe> (integer_to_list 1000 2)
"1111101000"
lfe> (integer_to_list 1000 8)
"1750"
lfe> (integer_to_list 1000 10)
"1000"
lfe> (integer_to_list 1000 16)
"3E8"
lfe> (integer_to_list 1000 36)
"RS"

If, for whatever reason, you want your base 10 integrer as a list (Erlang/LFE string), you can do that with this:

lfe> (integer_to_list 1000)
"1000"

Conversion to the binary type is also supported:

lfe> (integer_to_binary 1000)
#"1000"
lfe> (integer_to_binary 1000 2)
#"1111101000"
lfe> (integer_to_binary 1000 8)
#"1750"

The results above show LFE's literal representations of binary data; this will be covered in the chapter on "Bytes and Binaries".

Arithmetic Operators

Integers may be operated upon with the following:

lfe> (+ 1)
1
lfe> (+ 1 2)
3
lfe> (+ 1 2 3)
6
lfe> (- 1 2 3)
-4
lfe> (* 1 2 3)
6
lfe> (/ 1 2 3)
0.16666666666666666

Note that the division operator returns a float; floats will be covered in the next section.

Integer division is supported with a 2-arity function:

lfe> (div 1 2)
0
lfe> (div 10 2)
5

LFE also supports the remainder operation:

lfe> (rem 10 3)
1

As with any functional programming language, these operations may be composed (to any depth):

lfe> (div (* 12 (+ 1 2 3 4 5 6)) 6)
42

Mathematical Functions

The auto-loaded erlang module has several mathematical functions and is accessible in LFE without having to type the erlang: module prefix in the function calls. These include the following:

lfe> (abs -42)
42
lfe> (min 1 2)
1
lfe> (max 1 2)
2

Additional maths functions are provided via the math module. Since this module is not auto-loaded, in order to auto-complete it in the REPL you will need to load it:

lfe> (code:ensure_loaded 'math)
#(module math)

Now you can hit <TAB> after typing the following:

lfe> (math:

Which gives:

acos/1         acosh/1        asin/1         asinh/1        atan/1
atan2/2        atanh/1        ceil/1         cos/1          cosh/1
erf/1          erfc/1         exp/1          floor/1        fmod/2
log/1          log10/1        log2/1         module_info/0  module_info/1
pi/0           pow/2          sin/1          sinh/1         sqrt/1
tan/1          tanh/1

lfe> (round (math:pow 42 42))
150130937545296561928688012959677941476701514734130607701636390322176

The documentation for these functions is limited (available here) due in part to the fact that these are C library wrappers. Those that aren't documented should be self-explanatory for anyone who has used simular mathematical functions in other programming language libraries.

Predicates

To test if a value is an integer, we will first include some code:

lfe> (include-lib "lfe/include/cl.lfe")

That adds Common Lisp inspired functions and macros to our REPL session.

lfe> (integerp 42)
true
lfe> (integerp 42.24)
false
lfe> (integerp "forty-two")
false

If you prefer the Clojure-style of predicates:

lfe> (include-lib "lfe/include/clj.lfe")
lfe> (integer? 42)
true
lfe> (integer? "forty-two")
false

Of course there is always the Erlang predicate, usable without having to do any includes:

lfe> (is_integer 42)
true
lfe> (is_integer "forty-two")
false

Floats

Real numbers in LFE are represented using the stadnard floating point numbers.

lfe> 42.42
42.42
lfe> (/ 10 3)
3.3333333333333335
lfe> (math:pow 42 42)
1.5013093754529656e68

Note that the ~1.5e68 above is the floating point equivalent of scientific notation, namely 1.5 x 10⁶⁸. LFE follows the 64-bit standard for float representation given by IEEE 754-1985.

Converting

An integer may be converted to a float explicitly:

lfe> (float 42)
42.0

Or, as with integers, to binaries:

lfe> (float_to_binary 42.42)
#"4.24200000000000017053e+01"
lfe> (float_to_binary 42.42 '(#(scientific 10)))
#"4.2420000000e+01"
lfe> (float_to_binary 42.42 '(#(scientific 20)))
#"4.24200000000000017053e+01"
lfe> (float_to_binary 42.42 '(#(decimals 10)))
#"42.4200000000"
lfe> (float_to_binary 42.42 '(#(decimals 10) compact))
#"42.42"

Or to lists (LFE and Erlang strings):

lfe> (float_to_list 42.42 '(#(scientific 10)))
"4.2420000000e+01"
lfe> (float_to_list 42.42 '(#(scientific 20)))
"4.24200000000000017053e+01"
lfe> (float_to_list 42.42 '(#(decimals 10)))
"42.4200000000"
lfe> (float_to_list 42.42 '(#(decimals 10) compact))
"42.42"

Formatting

If you need to round floating point numbers to a specific precision, you'll want to use the format function from either the io, io_lib, or lfe_io modules. If just want to print a value using Erlang syntax and formatting, the io module is what you want. If you prefer LFE syntax and formatting for your output, you'll want to use the lfe_io module. If you want to use the data or store it in a variable, you'll need the io_lib library.

For default precision:

lfe> (io_lib:format "~f" `(,(math:pow 42 42)))
"150130937545296561929000000000000000000000000000000000000000000000000.000000"

Two decimal places:

lfe> (io_lib:format "~.2f" `(,(math:pow 42 42)))
"150130937545296561929000000000000000000000000000000000000000000000000.00"

20 decimal places:

lfe> (io_lib:format "~.20f" `(,(math:pow 42 42)))
"150130937545296561929000000000000000000000000000000000000000000000000.00000000000000000000"

Arithmetic Operators & Mathematical Functions

Floats use most of the same operators and functions as integers, so be sure to review these subsections in the "Integers" section.

Others include:

lfe> (abs -42)
42
lfe> (ceil 42.1)
43
lfe> (floor 42.1)
42
lfe> (round 42.4)
42
lfe> (round 42.5)
43
lfe> (min 1 2)
1
lfe> (max 1 2)
2

Predicates

To test if a value is an integer, we will first include some code:

lfe> (include-lib "lfe/include/cl.lfe")

That adds Common Lisp inspired functions and macros to our REPL session.

lfe> (floatp 42.42)
true
lfe> (floatp 42)
false
lfe> (floatp "forty-two.forty-two")
false

If you prefer the Clojure-style of predicates:

lfe> (include-lib "lfe/include/clj.lfe")
lfe> (float? 42.42)
true
lfe> (float? "forty-two.forty-two")
false

Of course there is always the Erlang predicate, usable without having to do any includes:

lfe> (is_float 42.42)
true
lfe> (is_float "forty-two.forty-two")
false

Atoms

The cloest analog in LFE to what most Lisp dialects call symbols is the Erlang atom. Just as with Lisps do with symbols, LFE uses atoms for its variable and function names. Atoms are literals and constants, which means that their value is the same as their name and once created, cannot be changed.

Some basic examples of atoms:

lfe> 'zark
zark
lfe> 'zarking-amazing
zarking-amazing

Slightly less straight-forward examples which start with non-alphanumeric characters:

lfe> ':answer
:answer
lfe> '42answer
42answer
lfe> '42.0e42answer
42.0e42answer
lfe> '42°C
42°C

Standard LFE atom names may be comprised of all the latin-1 character set except the following:

control character
whitespace
the various brackets
double quotes
semicolon

Of these, only |, \, ', ,, and # may not be the first character of the symbol's name (but they are allowed as subsequent letters).

Non-standard atom names may be created using atom quotes:

lfe> '|symbol name with spaces|
|symbol name with spaces|
lfe> '|'with staring quote!|
|'with staring quote!|
lfe> '|| ; <-- empty atoms are supported too
||
lfe> '|really weird atom: '#[]{}()<>";\||
|really weird atom: '#[]{}()<>";\||

In this case the name can contain any character of in the range from 0 to 255, and even no character at all.

In the case of atoms, it is important to understand a little something about their internals. In particular, Erlang and LFE atoms are global for each instance of a running Erlang virtual machine. Atoms are maintained by the VM in a table and are not garbage collected. By default, the Erlang atom table allows for a maximum of 1,048,576 entries.

Danger!

Uncontrolled autoamtic creation of atoms can crash the VM!

See the "Caveats" section below for more details.

As Symbols

The following code shows LFE's use of atoms in variable names. First, let's use a function for a slighly different purpose than designed: calling list_to_existing_atom on a string will only return a result if an atom of the same name already exists in the atom table. Otherwise, it will return an error:

lfe> (list_to_existing_atom "zaphod")
** exception error: bad argument
  in (erlang : list_to_existing_atom "zaphod")

This confirms that there is no zaphod atom in the atom table.

Now let's create a variable, assigning a value to it, and then use our indirect means of checkihg the atom table:

lfe> (set zaphod "frood")
"frood"
lfe> (list_to_existing_atom "zaphod")
zaphod

And here's an example showing LFE's use of atoms in function names using the same approach as above:

lfe> (list_to_existing_atom "beez")
** exception error: bad argument
  in (erlang : list_to_existing_atom "beez")

lfe> (defun beez (x) x)
beez
lfe> (list_to_existing_atom "beez")
beez

Converting

Atoms may be converted to strings and bitstrings, and vice versa.

(atom_to_binary 'trisha)
#"trisha"
(atom_to_binary 'mcmillan 'latin1)
#"mcmillan"
lfe> (atom_to_list 'trillian)
"trillian"

Note that the first one above is only available in Erlang 23.0 and above.

Some more examples for encoding:

lfe> (atom_to_binary '42°C 'latin1)
#B(52 50 176 67)
lfe> (atom_to_binary '42°C 'utf8)
#"42°C"

Functions that convert atoms only if they already exist in the atom table:

lfe> (binary_to_existing_atom #"trisha")
trisha
lfe> (binary_to_existing_atom #"trisha" 'latin1)
trisha
lfe> (list_to_existing_atom "trisha")
trisha

Operators

The only operators you may use on atoms are the comparison operators, e.g.:

lfe> (> 'a 'b)
false
lfe> (< 'a 'b)
true
lfe> (=:= 'a 'a)
true

Predicates

To test if a value is an atom, we will first include some code:

lfe> (include-lib "lfe/include/cl.lfe")

That adds Common Lisp inspired functions and macros to our REPL session.

lfe> (atomp 'arthur)
true
lfe> (atomp 42)
false
lfe> (atomp "forty-two.forty-two")
false

If the atom in question has been used in a function name definition:

If you prefer the Clojure-style of predicates:

lfe> (include-lib "lfe/include/clj.lfe")
lfe> (atom? 'dent)
true
lfe> (atom? "Ford")
false

Of course there is always the Erlang predicate, usable without having to do any includes:

lfe> (is_atom 'arthur)
true
lfe> (is_atom "forty-two.forty-two")
false

Caveats

As mentioned above (and documented), one needs to take care when creating atoms. By default, the maximum number of atoms that the Erlang VM will allow is 1,048,576; any more than that, and the VM will crash.

The first rule of thumb is not to write any code that generates large numbers of atoms. More explicitly useful, there are some handy functions for keeping track of the atom table, should you have the need.

lfe> (erlang:memory 'atom_used)
244562
lfe> (erlang:system_info 'atom_count)
9570
lfe> (erlang:system_info 'atom_limit)
1048576

Note that support for easily extracting the current atom data from system_info -- as demonstrated by the last two function calls above -- were added in Erlang 20; should you be running an older version of Erlang, you will need to parse the (system_info 'info) bitstring.

The default atom table size may be overridden during startup by passing a value with the +t options:

$ lfe +t 200000001

lfe> (erlang:system_info 'atom_limit)
200000001

Booleans

Strictly speaking, LFE has no Boolean type, just like Erlang. Instead, the atoms true and false are treated as Booleans

lfe> (== 42 "the question")
false
lfe> (== 42 42)
true
lfe> (> 1 2)
false
lfe> (< 1 2)
true

Operators

The standard logical operators are available to LFE Booleans:

lfe> (not 'true)
false
lfe> (not 'false)
true
lfe> (and 'true 'false)
false
lfe> (or 'true 'false)
true
lfe> (and 'true 'true 'true)
true
lfe> (and 'true 'true 'false)
false
lfe> (or 'false 'false 'false)
false
lfe> (or 'false 'false 'true)
true
lfe> (xor 'true 'true)
false
lfe> (xor 'true 'false)
true
lfe> (xor 'false 'false)
false
lfe> (xor 'false 'true)
true

With the and and or Boolean oprators, every argument is evaluated. To accomodate situations where complex and possible expensive logical expressions comprise the arguments to Boolean operators, short-circuit versions of these functions are also provided:

with andalso, returns as soon as the first 'false is encountered;
with orelse, returns as soon as the first 'true is encountered.

To demonstrate this, we'll define a boolean function that prints to standard out when it is evaluated:

(defun hey (x)
  (io:format "Made it here!~n") x)

Short-circuit demonstration of andalso:

lfe> (andalso 'true 'true 'false (hey 'true))
false
lfe> (andalso 'false 'true 'true (hey 'true))
false
lfe> (andalso 'true 'true 'true (hey 'true))
Made it here!
true

Short-circuit demonstration of orelse:

lfe> (orelse 'false 'false 'true (hey 'true))
true
lfe> (orelse 'true 'false 'false (hey 'true))
true
lfe> (orelse 'false 'false 'false (hey 'true))
Made it here!
true

Predicates

To test if a value is a Boolean, we will first include some code:

lfe> (include-lib "lfe/include/cl.lfe")

That adds Common Lisp inspired functions and macros to our REPL session.

lfe> (booleanp 'true)
true
lfe> (booleanp 'false)
true
lfe> (booleanp 'arthur)
false
lfe> (booleanp 42)
false

If the atom in question has been used in a function name definition:

If you prefer the Clojure-style of predicates:

lfe> (include-lib "lfe/include/clj.lfe")
lfe> (boolean? 'true)
true
lfe> (boolean? 'false)
true
lfe> (boolean? 'arthur)
false

Of course there is always the Erlang predicate, usable without having to do any includes:

lfe> (is_boolean 'true)
true
lfe> (is_boolean 'false)
true
lfe> (is_boolean 'arthur)
false

Note that, since LFE Booleans are also atoms, these are valid as well:

lfe> (atomp 'true)
true
lfe> (atom? 'false)
true
lfe> (is_atom 'true)
true

Characters

Characters in LFE Are represented internally by integers, however a literal syntax is offered for convenience:

lfe> #\a
97
lfe> #\A
65
lfe> #\ü
252
lfe> #\Æ
198

Converting

Since a character literal and integer are the same thing as far as LFE is concerned, there is no such thing as converting between a "char" and "ord" like there is in some other languages.

However, one can format an integer as a string by telling the class of format functions that the input is "character" type:

lfe> (io_lib:format "~c" `(198))
"Æ"

For merely printing to standard out instead of returning a value, one may use:

lfe> (lfe_io:format "~c~n" `(198))
Æ
ok
lfe> (io:format "~c~n" `(198))
Æ
ok

Operators

All operations that are valid for integers are valid for characters.

Predicates

All predicates that are valid for integers are valid for characters.

Primitive Types

This chapter covers the basic types of data available to LFE, upon which primatives rest all LFE libraries and applications.

Integers
Floats
Atoms
Booleans
Characters

Lists

Bytes and Binaries

Tuples

Property Lists

Maps

Arrays

Dicts

Records

Pattern Matching

Generic Sequence Functions

Part III - Data as Code

Expressions

Functions

Closures

Evaluation

Flow of Control

Processes

Messages and Their Passing

Objects and Flavors

I/O

Accessing Files

Modules

Packages

Scripting with LFE

Creating LFE Projects

Using rebar3

Project Layout Conventions

Part IV - Advanced Topics

Errors and Debugging

Writing Unit Tests

The Common Test Framework

The Propr Test Framework

The Compiler

Macros

Distributed LFE

Ports and Port Drivers

Servers

Clients

Part V - OTP

OTP Behaviours

Applications

Releases

Tables and Databases

Example OTP Project

Part VI - Tooling

rebar3

Quick Start

Introduction

This is a version of the LFE quick start that has been re-imagined with the LFE rebar3 plugin in mind.

We will cover the following:

How to get started quickly with LFE using just rebar3 and your local installation of Erlang
Creating a new LFE project
Looking at LFE code in the REPL and in modules
Provide a little insight on how this works
Leave you with resources for jumping into LFE in more detail

About `rebar3`

Rebar3 is an Erlang tool that makes it easy to create, develop, and release Erlang libraries, applications, and systems in a repeatable manner.

Rebar3 will:

respect and enforce standard Erlang/OTP conventions for project structure so they are easily reusable by the community;
manage source dependencies and Erlang packages while ensuring repeatable builds;
handle build artefacts, paths, and libraries such that standard development tools can be used without a headache;
adapt to projects of all sizes on almost any platform;
treat documentation as a feature, and errors or lack of documentation as a bug.

Rebar3 is also a self-contained Erlang script. It is easy to distribute or embed directly in a project. Tasks or behaviours can be modified or expanded with a plugin system flexible enough that even other languages on the Erlang VM will use it as a build tool.

The rebar3 site provides some nice instructions on installing rebar3.

Going Plaid

Remember: this is a quick-start; it's going to be a blur of colour! You're not going to get very many details here, but you will get to jump into the LFE REPL and see a little code in action.¹

The rest of this quick-start assumes that you've followed the links in the previous section and have installed both Erlang as well as rebar3, but to take things further, you'll need to do one more thing: set up the LFE plugin.

Each project you create with the LFE rebar3 plugin will generate a rebar.config file that automatically includes the plugin dependency, but that's only inside an LFE project. You need to bootstrap the LFE plugin by setting it up in your global rebar.config.

The rebar3 docs tell you this file is located at ~/.config/rebar3/rebar.config. To set this up, you can safely execute the following in a terminal, even if the file already exists:

mkdir -p ~/.config/rebar3/
touch ~/.config/rebar3/rebar.config

Then, in your preferred editor, open that file and add the entry for LFE rebar3 plugin. If that file is empty when you open it, then you can paste this whole thing in there:

{plugins, [
  {rebar3_lfe,
    {git, "https://github.com/lfe-rebar3/rebar3_lfe.git", {branch, "master"}}}
]}.

If you want to pin your project to a specific release of the plugin, you can view the list of released versions here:

https://github.com/lfe-rebar3/rebar3_lfe/tags

And then use tag (with the version) instead of branch:

{plugins, [
  {rebar3_lfe,
    {git, "https://github.com/lfe-rebar3/rebar3_lfe.git", {tag, "x.y.z"}}}
]}.

If your global rebar3 config file already has one or more plugins in it, then simply add a comma after the last one and paste the {rebar3_lfe ...} line from above (with no trailing comma!).

Next Stop

Ready for some LFE? Next you'll learn how to create a new LFE project with just one command ...

For those that would enjoy a more in-depth introduction and would appreciate having the time to see the stars (and not just stunning plaid), you may be interested in checking out The LFE Tutorial.

Creating a New Project

A project? Already?! It sounds daunting, but it's easier than you might think. Open up a terminal window and do this in a directory of your choosing:

rebar3 new lfe-lib my-test-lib

It might take a minute or two to finish; here's what's happening:

rebar3 downloads the LFE plugin
Finds its dependencies, downloads those too
Compiles all of them (plugins and dependencies)
rebar3 then executes the new command, searches for (and finds!) the lfe-lib template
Creates the project files for the given template

As that last step executes, you will see the following output:

===> Writing my-test-lib/README.md
===> Writing my-test-lib/LICENSE
===> Writing my-test-lib/rebar.config
===> Writing my-test-lib/.gitignore
===> Writing my-test-lib/src/my-test-lib.lfe
===> Writing my-test-lib/src/my-test-lib.app.src

It's as simple as that! Your new project is ready to go :-)

Next Stop

You can taste it, can't you? That LFE flavour coming your way? Yup, you're right. You're going to be looking at LFE code next ...

Hitting the Code

It may not seem like it, but we're off to a pretty fast start. If you had to do everything we've done, manually, you'd have given up by now. Seriously.

(Okay, maybe not.)

Time to put the brakes on, though, 'cause you're gonna want to see this next part in slow motion.

REPL Me Up!

Make sure you've cded into your new LFE project directory, and then do this:

$ rebar3 lfe repl

On windows first enter Erlang's repl then run lfe_shell:start().

D:\Lfe\my-test-lib
λ rebar3 lfe repl
===> Verifying dependencies...
===> Compiling my-test-lib
Eshell V10.3  (abort with ^G)
1> lfe_shell:start().

This should give you something that looks like the following:

Erlang/OTP 23 [erts-11.0.2] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [hipe] [dtrace]

   ..-~.~_~---..
  (      \\     )    |   A Lisp-2+ on the Erlang VM
  |`-.._/_\\_.-':    |   Type (help) for usage info.
  |         g |_ \   |
  |        n    | |  |   Docs: http://docs.lfe.io/
  |       a    / /   |   Source: http://github.com/rvirding/lfe
   \     l    |_/    |
    \   r     /      |   LFE v1.3-dev (abort with ^G)
     `-E___.-'

lfe>

Try setting a variable:

> (set my-list (lists:seq 1 6))
(1 2 3 4 5 6)

Here are some operations using more functions from the built-in Erlang lists module:

> (* 2 (lists:sum my-list))
42
> (* 2 (lists:foldl (lambda (n acc) (+ n acc)) 0 my-list))
42

Let's turn that into a function:

> (defun my-sum (start stop)
    (let ((my-list (lists:seq start stop)))
      (* 2 (lists:foldl
             (lambda (n acc)
               (+ n acc))
             0 my-list))))
my-sum

And try it out!

> (my-sum 1 6)
42

Enough with the fancy REPL-play ... What about some real code? What does a project look like?

Sample Code

Well, you've already seen some! But here is the full, if minimal, module generated by the LFE rebar3 plugin:

(defmodule my-test-lib
  (export (my-fun 0)))

;;; -----------
;;; library API
;;; -----------

(defun my-fun ()
  'hello-world)

You'll note that the function we define has been exported via the export form in the module definition. The number after the function is the arity of that function (Erlang views functions of the same name but different arity as different functions, and LFE does the same).

In the REPL you will have access to this module and its one function. Try it out:

lfe> (my-test-lib:my-fun)
hello-world

Let's add to this module our new my-sum function from the REPL jam session in the previous section. In another terminal window (or text editor pane) open up the src/my-test-lib.lfe file and paste the my-sum function at the bottom. Afterwards, add (my-sum 2) to the export section of defmodule at the top.

When you're done, the entire file should look like this:

(defmodule my-test-lib
  (export (my-fun 0)
          (my-sum 2)))

;;; -----------
;;; library API
;;; -----------

(defun my-fun ()
  'hello-world)

(defun my-sum (start stop)
  (let ((my-list (lists:seq start stop)))
    (* 2 (lists:foldl
            (lambda (n acc)
              (+ n acc))
            0 my-list))))

Then come back to the REPL sessions and compile the module with its new addition:

> (c "src/my-test-lib.lfe")
#(module my-test-lib)

And call the module functions:

> (my-test-lib:my-sum 1 6)
42
> (my-test-lib:my-sum 1 60)
3660
>

Here's something a little more involved you may enjoy, from the examples in the LFE source code:

(defun print-result ()
  (receive
    ((tuple pid msg)
      (io:format "Received message: '~s'~n" (list msg))
      (io:format "Sending message to process ~p ...~n" (list pid))
      (! pid (tuple msg))
      (print-result))))

(defun send-message (calling-pid msg)
  (let ((spawned-pid (spawn 'my-test-lib 'print-result ())))
    (! spawned-pid (tuple calling-pid msg))))

That bit of code demonstrates one of Erlang's core features in lovely Lisp syntax: message passing. When loaded into the REPL, that code can demonstrate bidirectional message passing between the LFE shell and a spawned process.

Want to give it a try? Add those two new functions to your module, and don't forget to update the export section, too! (note that one function has an arity of 0 and the other and arity of 2).

When you're done, your project module should look like this:

(defmodule my-test-lib
  (export (my-fun 0)
          (my-sum 2)
          (print-result 0)
          (send-message 2)))

;;; -----------
;;; library API
;;; -----------

(defun my-fun ()
  'hello-world)

(defun my-sum (start stop)
  (let ((my-list (lists:seq start stop)))
    (* 2 (lists:foldl
            (lambda (n acc)
              (+ n acc))
            0 my-list))))

(defun print-result ()
  (receive
    ((tuple pid msg)
      (io:format "Received message: '~s'~n" (list msg))
      (io:format "Sending message to process ~p ...~n" (list pid))
      (! pid (tuple msg))
      (print-result))))

(defun send-message (calling-pid msg)
  (let ((spawned-pid (spawn 'my-test-lib 'print-result ())))
    (! spawned-pid (tuple calling-pid msg))))

Message-Passing

We glossed over this in the previous section, but in LFE (and Erlang) you can compile on-the-fly in a REPL session. This is super-convenient when prototyping functionality for a new project where you want to use the REPL, but you also want the benefits of using a file, so you don't lose your work.

We made some changes to the sample code in the last section; let's compile it and take it for a spin:

> (c "src/my-test-lib.lfe")
#(module my-test-lib)

Next, let's send two messages to another Erlang process (in this case, we'll send it to our REPL process, (self):

> (my-test-lib:send-message (self) "And what does it say now?")
#(<0.26.0> "And what does it say now?")
Received message: 'And what does it say now?'
Sending message to process <0.26.0> ...
> (my-test-lib:send-message (self) "Mostly harmless.")
#(<0.26.0> "Mostly harmless.")
Received message: 'Mostly harmless.'
Sending message to process <0.26.0> ...

In the above calls, for each message sent we got a reply acknowledging the message (because the example was coded like that). But what about the receiver itself? What did it, our REPL process, see? We can flush the message queue in the REPL to find out.

What, what? Does each ...

Yup, every process in LFE (and Erlang, of course) has an inbox. You can see how many messages a given process has by looking at the process' info:

lfe> (erlang:process_info (self) 'message_queue_len)
#(message_queue_len 2)

And there you can see that our REPL process has two messages queued up in its inbox. Let's take a look!

> (c:flush)
Shell got {"And what does it say now?"}
Shell got {"Mostly harmless."}
ok

If you found this last bit interesting and want to step through a tutorial on Erlang's light-weight threads in more detail, you may enjoy this tutorial.

Next Stop

We did promise a bit more information, so we're going to do that next and then wrap up the quick start and point you in some directions for your next LFE adventures ...

Behind the Scenes

As we warned in the beginning, there's a lot going on behind the scenes: in rebar3 as well as LFE and Erlang (we haven't even touched OTP here ...!). This guide just gives you a quick taste of the LFE flavour :-) Before parting ways, though, there are some more bits we should share.

Some of those things are hinted at when just checking the current versions you are running using the LFE plugin's versions command (from a terminal where you have cded into your project directory):

rebar3 lfe versions

(#(apps (#(my-test-lib git)))
 #(languages
   (#(lfe "1.3-dev")
    #(erlang "23")
    #(emulator "11.0.2")
    #(driver_version "3.3")))
 #(tooling (#(rebar "3.10.0") #(rebar3_lfe "0.2.0"))))

To give a sense of what you'll encounter in the future: very often Erlang, LFE, and other BEAM language apps include more than just themselves when they are shipped. For instance, if you're in the REPL and you type (regs) you will see a list of applications that have been registered by name, currently running in support of the REPL. Usually, each app will have its own version. There is an LFE blog series on such things, showing you how to create and mess around with different types of LFE apps.

The LFE rebar3 plugin will also help you create OTP apps in LFE and perform other key tasks you may wish to integrate into your development workflow. You can learn more about those in the plugin's Command Reference

Next Stop

Where to go from here ...

Where Next?

We've mentioned the following resources so far:

But there are also these:

The old-skool LFE User Guide
Example code in the LFE repository
The lfex org - Community-contributed libraries
The lfe org - Core code, tools, and on-line resources (docs, blog, etc.)

True mastery of LFE is not matter of syntax, though: it requires a deep knowledge of Erlang/OTP and how to best apply that knowledge. The Erlang site has links to great Erlang books you can read.

Plugin Reference

Introduction

The rebar3_lfe project is a rebar3 plugin for the LFE language. It provides many of the conveniences one has come to expect of a programming language's build tool:

Project Creation
A REPL
Compilation
Maintenane Tasks (e.g., file cleanup)
Metadata

Features

Create new LFE projects:
- rebar3 new lfe-lib
- rebar3 new lfe-main
- rebar3 new lfe-escript
- rebar3 new lfe-app
- rebar3 new lfe-release
Start up an LFE REPL:
- rebar3 lfe repl
Compile LFE source code:
- rebar3 lfe compile
Run eunit tests
- rebar3 eunit
Run an LFE project's main/1 function as an lfescript (run rebar3 new lfe-main to see an example):
- rebar3 lfe run
- rebar3 lfe run -- 1 2 5
- rebar3 lfe run -main some/path/main.lfe
Escriptize an LFE escript project:
- rebar3 ecsriptize
Run an escriptized LFE escript project:
- rebar3 lfe run-ecsript
Generate an LFE/OTP release
- rebar3 release
Run an LFE/OTP release project's release script (COMMAND can be start , stop , status, ping, etc.):
- rebar3 lfe run-release COMMAND
Cleanup
- rebar3 lfe clean
- rebar3 lfe clean-build
- rebar3 lfe clean-cache
- rebar3 lfe clean-all
Metadata
- rebar3 lfe versions

More coming soon!

Background

This plugin originally started life as a shell script (lfetool -- there's even a T-shirt for it!), then it toyed with integrating with rebar (the original). Around that time, though, rebar3 was under initial development, and LFE took a chance on it as an early adopter. This lead to a whole series of LFE plugins, but after a few years momentum was lost.

Those early rebar3 efforts have been combined into a single plugin in this project, with many updates and using all the latest approaches developed in rebar3's now mature ecosystem.

Setup

Dependencies

In order to use the LFE rebar3 plugin, you need to have the following installed on your system:

Erlang (only tested with versions 19 and above)
rebar3 (tested with 3.10 and 3.12)

You don't need to download LFE; the plugin will do that for you.

Using the Plugin

After installing rebar3, the only thing you need to do in order to take full advantage of the LFE rebar3 plugin is add it to the plugins in your global rebar.config file.

Stable

To use the latest stable release, update your `rebar.config` to:

{plugins, [
  {rebar3_lfe,
    {git, "https://github.com/lfe-rebar3/rebar3_lfe.git", {tag, "0.2.0"}}}
]}.

Unstable

If you want to use the current development branch (unstable):

{plugins, [
  {rebar3_lfe,
    {git, "https://github.com/lfe-rebar3/rebar3_lfe.git", {branch, "release/0.3.x"}}}
]}.

Command Reference

The following sections detail actual usage of the suite of rebar3 lfe commands.

Compiling

The single most imporant convenience provided by the LFE rebar3 plugin is arguably the compiler. This allows any LFE project to be downloaded, compile, and used by any BEAM language that is also using rebar3 to manage its dependencies, etc.

To compile an LFE project:

rebar3 lfe compile

If you are publishing your LFE code, or using it in another project, you'll want to update your rebar.config file so that it is compile when a user (or script) executes the regular rebar3 compile command.

To ensure your LFE code will compile in other projects, add the following to your project's rebar.config:

{provider_hooks, [
  {pre, [{compile, {lfe, compile}}]}
  ]}.

Package Support

The LFE rebar3 plugin provides support for pseudo-packages. There is no such thing as a pckage in Erlang, but using this plugin, you can emulate some of the behaviours of packages.

This is accomplished by traversing top-level source directories for any subdirectories: if the plugin finds any .lfe or .erl files in subdirectories under configured source directories, it will create a dotted module name composed of the relative path to that file, and write that name to the ebin directory after successful compilation.

Here are some examples of how combinations of subdirectories and files will be transformed in their final form as .beam files:

./src/my.package1.lfe                      -> ebin/my.package1.beam
./src/my/package2.lfe                      -> ebin/my.package2.beam
./src/my/other/package.lfe                 -> ebin/my.other.package.beam
./src/my/really/deeply/nested/package1.lfe -> ebin/my.really.deeply.nested.package1.beam
./src/my/really/deeply/nested.package2.lfe -> ebin/my.really.deeply.nested.package2.beam
./src/my/really.deeply.nested.package3.lfe -> ebin/my.really.deeply.nested.package3.beam
./src/my.really.deeply.nested.package4.lfe -> ebin/my.really.deeply.nested.package4.beam

Running the REPL

An LFE project and all of its dependencies may be interacted with via a REPL that is started with the LFE rebar3 plguin, as rebar3 sets all of the correct library locations for use by shells and REPLs.

To start up a REPL:

rebar3 lfe repl

At which point you will be greeted with the familiar:

Erlang/OTP 25 [erts-13.2.2.2] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]

   ..-~.~_~---..
  (      \\     )    |   A Lisp-2+ on the Erlang VM
  |`-.._/_\\_.-':    |   Type (help) for usage info.
  |         g |_ \   |
  |        n    | |  |   Docs: http://docs.lfe.io/
  |       a    / /   |   Source: http://github.com/lfe/lfe
   \     l    |_/    |
    \   r     /      |   LFE v2.1.2 (abort with ^C; enter JCL with ^G)
     `-E___.-'

lfe>

Known Issue: Erlang 26+!

Erlang 26.0 completely refactored its support for shells. While a fix was released for LFE-proper's REPL, that same fix does not work for the rebar3 LFE REPL. To follow the progress, you can subscribe to this ticket: https://github.com/lfe/rebar3/issues/79.

The LFE banner is highly configurable in in rebar3_lfe and accepts the following configuration in your project's rebar.config file:

{lfe, [
   {repl, [
     {nobanner, false},
     {version, "9999.2"},
     {quit_message, "You can never LEEEEEAVE!"},
     {banner_template, "Weeee!~nLFE v~s ~s~n"}
   ]}
]}.

A project configuration with those values would have a banner like the following:

Erlang/OTP 25 [erts-13.2.2.2] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]

Weeee!
LFE v9999.2 You can never LEEEEEAVE!
lfe>

To disable the banner, use a configuration like this:

{lfe, [
   {repl, [
     {nobanner, true}
   ]}
]}.

Which would give:

Erlang/OTP 25 [erts-13.2.2.2] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]

lfe>

Testing

[In progress]

eunit

If you have written eunit tests in Erlang or LFE, they will be compiled by either the rebar3 compile or rebar3 lfe compile command and thus be available and ready to run using rebar3 as-is.

Run compiled eunit tests:

rebar3 eunit

ltest

[In progress]

Common Test

[In progress]

Creating Projects

The rebar3_lfe plugin is capable of creating several common project layouts. This and following sections provide details on those that are currently supported.

Creating LFE Libraries

Library projects are those with no running applications or scripts; they simply provide some core bit of functionality intended for use by applications, scripts, or other libraries.

To create a library project with the default name:

rebar3 new lfe-lib

This will generate the following output:

===> Writing my-lfe-lib/README.md
===> Writing my-lfe-lib/LICENSE
===> Writing my-lfe-lib/rebar.config
===> Writing my-lfe-lib/.gitignore
===> Writing my-lfe-lib/src/my-lfe-lib.lfe
===> Writing my-lfe-lib/src/my-lfe-lib.app.src

You can also explicitely name your project:

rebar3 new lfe-lib forty-two

Which will produce the following:

===> Writing forty-two/README.md
===> Writing forty-two/LICENSE
===> Writing forty-two/rebar.config
===> Writing forty-two/.gitignore
===> Writing forty-two/src/forty-two.lfe
===> Writing forty-two/src/forty-two.app.src

As mentioned abouve, the REPL offers a nice way to quickly interact your new project.

Start the REPL:

cd forty-two
rebar3 lfe repl

Call the generated/sample LFE function:

lfe> (mything:my-fun)
hello-world

Creating `main` Scripts

LFE supports the creation of scripts, and these can be integrated with libraries, allowing for the best of both worlds: a simple collection of useful functions (library) and a means of running them from the command line.

Create a library that also has a script with a main function:

rebar3 new lfe-main special-proj

Which generates:

===> Writing special-proj/README.md
===> Writing special-proj/LICENSE
===> Writing special-proj/rebar.config
===> Writing special-proj/.gitignore
===> Writing special-proj/src/special-proj.lfe
===> Writing special-proj/scripts/main.lfe
===> Writing special-proj/src/special-proj.app.src

The generated project's main script + function may then be run with:

cd mymain
rebar3 lfe run -- 42

Which will produce the following output:

Running script '/usr/local/bin/rebar3' with args [<<"42">>] ...
'hello-worl

Creating escripts

The LFE rebar3 plugin also supports generating escript-based projects in LFE. This is similar in nature to the main-based project, but is more standard in the BEAM family of languages.

To create an escript-based project:

rebar3 new lfe-escript myapp

===> Writing myapp/README.md
===> Writing myapp/LICENSE
===> Writing myapp/rebar.config
===> Writing myapp/.gitignore
===> Writing myapp/src/myapp.lfe
===> Writing myapp/src/myapp.app.src

Compile the LFE and then bundle all the code up by "escriptizing" it:

cd myapp
rebar3 lfe compile
rebar3 escriptize

Run the newly-created escript:

rebar3 lfe run-escript 1 2 5 no '3!'

Which will display the following:

Got args: ("1" "2" "5" "no" "3!")
Answer: 42

Creating OTP Applications

In the wider Erlang community, it is very common to see applications that run one or more gen_servers (or other server behaviours) managed by a supervision tree (with the appropriate restart strategy defined). The LFE rebar3 plugin provides the ability to generate OTP gen_server applications with the server managed in a supervision tree.

To create an LFE/OTP application:

rebar3 new lfe-app otp-lfe

===> Writing otp-lfe/README.md
===> Writing otp-lfe/LICENSE
===> Writing otp-lfe/rebar.config
===> Writing otp-lfe/.gitignore
===> Writing otp-lfe/src/otp-lfe.lfe
===> Writing otp-lfe/src/otp-lfe-app.lfe
===> Writing otp-lfe/src/otp-lfe-sup.lfe
===> Writing otp-lfe/src/otp-lfe.app.src

We can use the plugin's REPL command to demonstrate usage.

Start the REPL:

cd otp-lfe
rebar3 lfe repl

Start the app:

lfe> (application:ensure_all_started 'otp-lfe)
#(ok (otp-lfe))

Make sure the supervisor is running:

lfe> (erlang:whereis 'otp-lfe-sup)
#Pid<0.205.0>

Make an API call to the gen_server:

lfe> (otp-lfe:echo "testing the supervised gen_server ...")
"testing the supervised gen_server ..."

Creating OTP Releases

If you're going to be running an LFE application in production, you will very likely want to do so using the "release" functionality provided by OTP.

Create a release-based project with:

rebar3 new lfe-release prod-lfe

===> Writing prod-lfe/README.md
===> Writing prod-lfe/LICENSE
===> Writing prod-lfe/rebar.config
===> Writing prod-lfe/.gitignore
===> Writing prod-lfe/apps/prod-lfe/src/prod-lfe.lfe
===> Writing prod-lfe/apps/prod-lfe/src/prod-lfe-app.lfe
===> Writing prod-lfe/apps/prod-lfe/src/prod-lfe-sup.lfe
===> Writing prod-lfe/apps/prod-lfe/src/prod-lfe.app.src
===> Writing prod-lfe/config/sys.config
===> Writing prod-lfe/config/vm.args

Change directoy into your new app:

cd prod-lfe

Build the release:

rebar3 release

Start up the application:

rebar3 lfe run-release start

Check the status of the application:

rebar3 lfe run-release ping

pong

Known Issue!

If your ping doesn't get a pong after starting the release, this is a known issue that is being investigated in the following ticket:

https://github.com/lfe-rebar3/rebar3_lfe/issues/33

Workaround

The current workaround for a relese that doesn't start is simply to run the following again:

rebar3 release
rebar3 lfe run-release start
rebar3 lfe run-release ping

In addition to using the LFE rebar3 commands to start the application, you can start up a release console and then switch to the LFE REPL.

Start the console:

./_build/default/rel/prod-lfe/bin/prod-lfe console

Eshell V11.0  (abort with ^G)
(prod-app@spacemac)1> lfe_shell:start().

(prod-app@spacemac)lfe> (erlang:whereis 'prod-lfe-sup)
#Pid<0.351.0>
(prod-app@spacemac)lfe> (prod-lfe:echo "testing from production!")
"testing from production!"

Running Code

The LFE rebar3 plugin provides developers the ability to run LFE code directly, taking advantage of rebar3's setting of library paths for dependencies, etc.

run

To run an LFE project that was generated with rebar3 new lfe-main:

rebar3 lfe run

run-escript

To run an LFE project that was generated with rebar3 new lfe-escript:

rebar3 lfe run-escript

run-release

To run an LFE project that was generated with rebar3 new lfe-release and has been compiled with rebar3 release:

rebar3 lfe run-release COMMAND

Where COMMAND is one of the non-interactive commands supported by the release script:

start
stop
restart
reboot
pid
ping

Others non-interactive commands not listed may also work, but they have not been tested.

Known Issue!

If your release doesn't start (e.g., running rebar3 lfe run-release ping doesn't return pong), this is a known issue that is being investigated in the following ticket:

https://github.com/lfe-rebar3/rebar3_lfe/issues/33

Workaround

The current workaround for a relese that doesn't start is simply to re-run rebar3 release and rebar3 lfe run-release start.

Cleanup Commands

There are a handful of tasks the author of the plugin has found useful when debugging LFE applications, rebar3 plugins, etc. These include various "cleanup" tasks that are not currently supported by rebar3 (or whose support is problematic).

clean

Remove the apps' ebin/* files:

rebar3 lfe clean

clean-build

Remove the _build directory:

rebar3 lfe clean-build

clean-cache

Remove the the cached directories for the app and the rebar3_lfe plugin, both global and local:

rebar3 lfe clean-cache

clean-all

Perform all clean tasks as well as remove the files erl_crash.dump, rebar3.crashdump, and rebar.lock:

rebar3 lfe clean-all

Part VII - Guides

Style Guide

Introduction

What is good style?

Good style in any language consists of code that is:¹

Understandable
Reusable
Extensible
Efficient
Easy to develop and debug

It also helps ensure correctness, robustness, and compatibility. Maxims of good style are:

Be explicit
Be specific
Be concise
Be consistent
Be helpful (anticipate the reader's needs)
Be conventional (don't be obscure)
Build abstractions at a usable level
Allow tools to interact (referential transparency)

Know the context when reading code:

Who wrote it and when?
What were the business needs?
What other factors contributed to the design decisions?

Sources and Inspiration

The LFE Style Guide takes inspiration (and often times actual content) directly from key sources in the Lisp, Erlang, and even Clojure developer communities. These are as follows

Note, however, that these are not considered sacrosanct sources of ultimate truth; (and neither is this guide). Instead, they contain practices that we have either adopted as-is, modified to some extent, or simply rejected (e.g., due to prior conventions established in MACLISP and LMI Lisp, their inapplicability due to LFE's unique status as a Lisp and Erlang dialect, etc.).

In general we suggest following the LFE style as outlined here if you are creating a new project. If you are contributing to a project maintained by someonoe in the community, we recommend consistency: using the style adopted by that project (for any contributions to that project).

Above all, enjoy the parenthesis.

Notes

¹ This section was adapted from the Tutorial on Good Lisp Programming Style by Peter Norvig and Kent Pitman.

Formatting

Topics related to the manner of formatter LFE code.

File Headers

Every source file should begin with a brief description of the contents of that file.

After that description, every file should start the code itself with a (defmodule ...) form.

;;;; Variable length encoding for integers and floating point numbers.

(defmodule num-encode
  ...)

It is not necessary to include copyright info in every file as long as the project has a LICENSE file in its top-level directory. Files which differ in license from that file should get have a copyright notice in their header section.

If you are contributing to a project that has established a convention of adding copyright headers to all files, simply follow that convention.

Indentation

In general, use your text editor's indentation capabilities. If you are contributing to a particular library, be sure to ask the maintainers what standard they use, and follow those same guidelines, thus saving everyone from the drudgery of whitespace fixes later.

In particular, you'll want to do everything you can to follow the conventions laid out in the Emacs LFE mode supplied in the LFE source. Instructions for use are given in the LFE Github wiki, but we'll repeat it here. Simply edit your ~/.emacs file to include the following:

;; Prevent tabs being added:
(setq-default indent-tabs-mode nil)

;; LFE mode.
;; Set lfe-dir to point to where the lfe emacs files are.
(defvar lfe-dir (concat (getenv "HOME") "/git/lfe/emacs"))
(setq load-path (cons lfe-dir load-path))
(require 'lfe-start)

In general though, indentation is two columns per form, for instance:

(defun f ()
  (let ((x 1)
        (y 2))
    (lfe_io:format "X=~p, Y=~p~n"  (list x y))))

Note that LFE has many exceptions to this rule, given the complex forms it defines for features inherited from Erlang (e.g., pattern-matching in function heads). A few examples for the number exceptions to the two-space indentation rule above:

(cond ((lists:member x '(1 2 3)) "First three")
      ((=:= x 4) "Is four")
      ((>= x 5) "More than four")
      ('true "You chose poorly"))

(defun ackermann
  ((0 n)
   (+ n 1))
  ((m 0)
   (ackermann (- m 1) 1))
  ((m n)
   (ackermann (- m 1) (ackermann m (- n 1)))))

The last function would actually be better written as follows, but the form above demonstrates the indentation point:

(defun ackermann
  ((0 n) (+ n 1))
  ((m 0) (ackermann (- m 1) 1))
  ((m n) (ackermann (- m 1) (ackermann m (- n 1)))))

Maintain a consistent indentation style throughout a project.

Indent carefully to make the code easier to understand.

Use indentation to make complex function applications easier to read. When an application does not fit on one line or the function takes many arguments, consider inserting newlines between the arguments so that each one is on a separate line. However, do not insert newlines in a way that makes it hard to tell how many arguments the function takes or where an argument form starts and ends.

Bad:

(do-something first-argument second-argument (lambda (x)
    (frob x)) fourth-argument last-argument)

Better:

(do-something first-argument
              second-argument
              (lambda (x) (frob x))
              fourth-argument
              last-argument)

Vertical White Space

You should include one blank line between top-level forms, such as function definitions. Exceptionally, blank lines can be omitted between simple, closely related defining forms of the same kind, such as a group of related type declarations or constant definitions.

(defun +my-pi+ () 3.14)
(defun +my-e+ () 2.72)

(defun factorial (n)
  (factorial n 1))

(defun factorial
  ((0 acc) acc)
  ((n acc) (when (> n 0))
   (factorial (- n 1) (* n acc))))

Horizontal White Space

Do not include extra horizontal whitespace before or after parentheses or around symbols.

Furthermore, do not place right parentheses by themselves on a line. A set of consecutive trailing parentheses must appear on the same line.

Very bad:

( defun factorial
  (
    ( 0 acc)
    acc
  )
  (
    ( n acc)
      ( when ( > n 0)
  )
  ( factorial ( - n 1)
    ( * n acc
       )
     )
  )
)

Much better:

(defun factorial
  ((0 acc) acc)
  ((n acc) (when (> n 0))
   (factorial (- n 1) (* n acc))))

You should use only one space between forms.

You should not use spaces to vertically align forms in the middle of consecutive lines. An exception is made when the code possesses an important yet otherwise not visible symmetry that you want to emphasise.

Bad:

(let* ((low    1)
       (high   2)
       (sum    (+ (* low low) (* high high))))
  ...)

Better:

(let* ((low 1)
       (high 2)
       (sum (+ (* low low) (* high high))))
  ...))

You should align nested forms if they occur across more than one line.

Bad:

(defun munge (a b c)
(* (+ a b)
c))

Better:

(defun munge (a b c)
  (* (+ a b)
     c))

Line Length

You should format source code so that no line is longer than 80 characters.

Old text terminals were standardised on 80 columns which they in turn inherited from even older punch card technology. While modern computer screens support vastly more than this, there are a couple of considerations to keep in mind that motivate us to continue supporting an 80 character limit:

Displaying code in web pages, paste-bins, gist services, etc., is much cleaner and easier to read when the character width is limited to 80 characters.
Most modern text editors allow for multiple panes, allowing several files to be open side-by-side, supporting the easy editing and referencing of multiple files simultaneously; limiting these files to 80 characters in width facilitates this type of workflow.
Code that has to be examined under emergency circumstances (such as via a terminal attached to a crash cart in a data centre, or in an emergency shell session without a graphical window manager) is much easier to read quickly when character width is limited to 80.
Lastly, such a convention encourages good naming discipline!

Spelling and Abbreviations

Use correct spelling in your comments, and most importantly in your identifiers. The LFE documentation projects (books and reference materials) use aspell and include make targets for running various spell-checking tasks across the project files. Feel free to borrow from these for your own projects.

Use common and domain-specific abbreviations, and must be consistent with these abbreviations. You may abbreviate lexical variables of limited scope in order to avoid overly-long symbol names.

If you're not sure, consult a dictionary, look up alternative spellings in a dictionary, or ask a local expert.

Here are examples of choosing the correct spelling:

Use "complimentary" in the sense of a meal or beverage that is not paid for by the recipient, not "complementary".
Use "existent" and "nonexistent", not "existant". Use "existence", not "existance".
Use "hierarchy" not "heirarchy".
Use "precede" not "preceed".
Use "weird", not "wierd".

Make appropriate exceptions for industry standard nomenclature/jargon, including plain misspellings. For instance:

Use "referer", not "referrer", in the context of the HTTP protocol.

Naming

On names and naming in LFE code.

Symbols

Use lower case for all symbols (Erlang "atoms"). Consistently using lower case makes searching for symbol names easier and is more readable.

Place hyphens between all the words in a symbol. If you can't easily say an identifier out loud, it is probably badly named.

Always prefer - over / or . unless you have a well-documented overarching reason to, and approval from other hackers who review your proposal.

Bad:

(defun *default-username* ()"Ann")
(defun *max-widget-cnt* () 200)

Better:

(defun *default-user-name* () "Ann")
(defun *maximum-widget-count* () 200)

Unless the scope of a variable is very small, do not use overly short names like i and zq.

Names in Modules

When naming a symbol in a module, you should not include the module name as part of the name of the symbol. Naming a symbol this way makes it awkward to use from a client module accessing the symbol by qualifying it with a module prefix, where the module name then appears twice (once as part of the module name, another time as part of the symbol name itself).

Bad:

(defmodule varint
 (export
   (varint-length64 0))

(defun varint-length64 () ... )

(defmodule client-code)

(defun +padding+ ()
 (varint:varint-length64))

Better:

(defmodule varint
  (export
    (length64 0))

(defun length64 () ... )

(defmodule client-code)

(defun +padding+ ()
  (varint:length64))

Global Variables and Constants

Erlang, and thus LFE, does not support global variables or mutable data. However, many projects define constants in modules. Traditionally, Lisp projects have used symbols enclosed in + for global constants and symbols enclosed in * (a.k.a. "earmuffs") for global variables.

Adapted for LFE, one could use these conventions for module constants and default values, respectively.

(defun +my-pi+ () 3.14)
(defun *default-host* () "127.0.0.1")

Predicates

There are several options for naming boolean-valued functions and variables to indicate they are predicates:

a trailing ?
a trailing -p
a trailing p
a leading is-

Modern Lisps tend to prefer ?, while classic Lisps tend to use p. Erlang code tends to use is_ which translates to is- in LFE. You should use "p" when the rest of the function name is one word and "-p" when it is more than one word.

A rationale for this convention is given in the CLtL2 chapter on predicates.

Whichever convention your project wishes to use, be consistent and use only that convention in the entire project.

Do not use these boolean indicators in functions that do not return booleans and variables that are not boolean-valued.

Intent not Content

You should name a variable according to the high-level concept that it represents (intent), not according to the low-level implementation details of how the concept is represented (content).

Thus, you should avoid embedding data structure or aggregate type names, such as list, array, or hash-table as part of the variable names, unless you're writing a generic algorithm that applies to arbitrary lists, arrays, hash-tables, etc. In that case it's perfectly OK to name a variable list or array.

For example, if a variable's value is always a row (or is either a row or NIL), it's good to call it row or first-row or something like that.

Be consistent. If a variable is named row in one function, and its value is being passed to a second function, then call it row rather than, say, value.

Documentation

Topics related to in-code LFE documentation.

Docstrings

First and foremost, document everything.

You should use document strings (a.k.a. "docstrings") on all visible functions to explain how to use your code.

Unless some bit of code is painfully self-explanatory, document it with a documentation string.

Documentation strings are destined to be read by the programmers who use your code. They can be extracted from functions, types, classes, variables and macros, and displayed by programming tools, such as IDEs, or by REPL queries; web-based documentation or other reference works can be created based on them. Documentation strings are thus the perfect locus to document your API. They should describe how to use the code (including what pitfalls to avoid), as opposed to how the code works (and where more work is needed), which is what you'll put in comments.

Supply a documentation string when defining top-level functions, records, classes, variables and macros. Generally, add a documentation string wherever the language allows.

For functions, the docstring should describe the function's contract: what the function does, what the arguments mean, what values are returned, what conditions the function can signal. It should be expressed at the appropriate level of abstraction, explaining the intended meaning rather than, say, just the syntax.

Some LFE forms do not accept docstrings, in which case a preceding code comment should be used instead.

(defun small-prime-number? (n)
  "Return true if N, an integer, is a prime number. Otherwise, return false."
  ((n) (when (< n 4))
   (>= n 2))
  ((n) (when (== 0 (rem n 2)))
   'false)
  ((n)
   (lists:all #'not/1
              (lists:map (lambda (x) (== 0 (rem n x)))
                         (lists:seq 3 (trunc (math:sqrt n)))))))

(defmacro is (bool-expression)
  "Assert bool-expression evaluates to 'true."
  `(assert ,bool-expression))

;;; This record tracks test results and is ulimately used when reporting the
;;; status of completed tests.
(defrecord state
  (status (orddict:new))
  test-type
  (ok 0)
  (fail 0)
  (err 0)
  (skip 0)
  (cancel 0)
  (time 0))

A long docstring may usefully begin with a short, single-sentence summary, followed by the larger body of the docstring.

Text in successive lines of docstrings are indented two spaces, aligned with the open quote in the first line of documentation, not with the first character of the text.

Code Comments

Comments are explanations to the future maintainers of the code. Even if you're the only person who will ever see and touch the code, even if you're either immortal and never going to quit, or unconcerned with what happens after you leave (and have your code self-destruct in such an eventuality), you may find it useful to comment your code. Indeed, by the time you revisit your code, weeks, months or years later, you will find yourself a different person from the one who wrote it, and you will be grateful to the previous you for making the code readable.

Comment anything complicated so that the next developer can understand what's going on.

Also use comments as a way to guide those who read the code, so they know what to find where.

Code comments in LFE, as in most Lisp dialects, begin with a semi-colon, with their number having conventional semantic value:

Four Semi-colons: These are used for file headers and important comments that apply to large sections of code in a source file.
Three Semi-colons: These are used to begin comments that apply to just one top-level form or small group of top-level forms.
Two Semi-colons: These are used inside a top-level form, for comments appearing between lines. For code that uses unobvious forms to accomplish a task, you must include a comment stating the purpose of the form and the task it accomplishes.
One Semi-colon: This is used for parenthetical remark and only occurs at the end of a line. You should use spaces to separate the comment from the code it refers to so the comment stands out. You should try to vertically align consecutive related end-of-line comments.

For all comments, there should be a space between the semicolon and the text of the comment.

;;;; File-level comments or comments for large sections of code.
(defmodule math-n-things
  (export
   (utility-function 0)
   ...
   (small-prime-number? 1)
   (large-prime-number? 1)
   ...))

;;; The functions in this section are utility in nature, supporting others in
;;; the module. More details on their intended use cases are available here:
;;; * https://some.url/

(defun utility-function ()
  ...)

;;; Prime numbers section

(defun small-prime-number? (n)
  "Return true if N, an integer, is a prime number. Otherwise, return false."
  ((n) (when (< n 4))  ; parenthetical remark here
   (>= n 2))           ; continuation of the remark
  ((n) (when (== 0 (rem n 2)))
   'false)             ; different remark here
  ((n)
   ;; Comment that applies to a section of code.
   (lists:all #'not/1
              (lists:map (lambda (x) (== 0 (rem n x)))
                         (lists:seq 3 (trunc (math:sqrt n)))))))

(defun large-prime-number? (n)
  ...)

Attention Required

For comments requiring special attention, such as incomplete code, todo items, questions, breakage, and danger, include a TODO or XXX comment indicating the type of problem, its nature, and any notes on how it may be addressed.

The comments begin with TODO or XXX in all capital letters, followed by the name, e-mail address, or other identifier of the person with the best context about the problem referenced by the TODO or XXX. The main purpose is to have a consistent TODO or XXX that can be searched to find out how to get more details upon request. A TODO or XXX is not a commitment that the person referenced will fix the problem. Thus when you create a TODO or XXX, it is almost always your name that is given.

Generally, TODO and XXX commands are differentiated in that TODO items represent normal code tasks around such things as incomplete features and XXX items represent a bug, potential bug, pitfalls, incorrectness, inelegance, uncertainty about part of the code, etc. Common synonyms for XXX include BUG, FIXME and sometimes HACK (this last especially for incorrectness or inelegance).

When signing comments, you should use your username (for code within the company) or full email address (for code visible outside the company), not just initials.

;; --- TODO (alice@gmail.com): Refactor to provide a better API.

;; --- TODO (bob): Remove this code after release 1.7 or before 2012-11-30.

If there is an associated issue or bug ticket with the given TODO or XXX item, be sure to include that in a following line:

;; --- XXX (carol): There is a serious issue here, causing problems in other
;;                  areas of the code. We haven't decided upon the best
;;                  approach yet. See the following ticket for details:
;;
;;                   * https://github.com/examplecom/api/issues/42
;;

Data Representation

Notes on basic data structures.

Lists

Use the appropriate functions when manipulating lists.

For simple access to list data, you can use car, cdr, cadr, etc. to access list elements and segments. For common pattern-matching in function heads, receive, let, etc., use cons to access the head and tail of a list (e.g., (,head . ,tail)).

Additionally, don't forget the lists Erlang module for accessing list elements.

You should avoid using a list as anything besides a container of elements of like type. Avoid using a list as method of passing multiple separate values of different types in and out of function calls. Sometimes it is convenient to use a list as a little ad hoc structure, i.e. "the first element of the list is a foo, and the second is a bar", but this should be used minimally since it gets harder to remember the little convention. You should only use a list that way when destructuring the list of arguments from a function, or creating a list of arguments to which to apply a function.

The proper way to pass around an object comprising several values of heterogeneous types is to use a record created via defrecord.

Tuples and Proplists

Do not align keys and values in property lists; instead, simply use the standard Lisp formatting (e.g, as provided by the LFE Emacs formatter).

Bad:

'(#(k1            v1)
  #(key2          value2)
  #(key-the-third value-the-third)
  #(another       one))

Better:

'(#(k1 v1)
  #(key2 value2)
  #(key-the-third value-the-third)
  #(another one))

Maps

Do not align keys and values in maps. Note, however, that the LFE Emacs formatter doesn't currently indent maps properly.

Bad:

'#m(k1            v1
    key2          value2
    key-the-third value-the-third
    another       one)

Also bad (formatted with the LFE Emacs formatter):

'#m(k1 v1
       key2          value2
       key-the-third value-the-third
       another       one)

Better:

#m(k1 v1
   key2 value2
   key-the-third value-the-third
   another one)

Records

Use records as the principle data structure

Use records as the principle data structure in messages. A record is a tagged tuple and was introduced in Erlang version 4.3 and thereafter.

If the record is to be used in several modules, its definition should be placed in a header file (with suffix .lfe) that is included from the modules. If the record is only used from within one module, the definition of the record should be in the beginning of the file the module is defined in.

The record features of LFE can be used to ensure cross module consistency of data structures and should therefore be used by interface functions when passing data structures between modules.

Use selectors and constructors

Use the record macros provided by LFE for managing instances of records. Don't use matching that explicitly assumes that the record is a tuple.

Bad:

(defun demo ()
  (let* ((joe (make-person name "Joe" age 29))
         (`#(person ,name ,age) joe))
    ...
    ))

Good:

(defun demo ()
  (let* ((joe (make-person name "Joe" age 29))
         (name-2 (person-name joe)))
    ...
    ))

Errors

Separate error handling and normal case code

Don't clutter code for the "normal case" with code designed to handle exceptions. As far as possible you should only program the normal case. If the code for the normal case fails, your process should report the error and crash as soon as possible. Don't try to fix up the error and continue. The error should be handled in a different process.

Clean separation of error recovery code and normal case code should greatly simplify the overall system design.

The error logs which are generated when a software or hardware error is detected will be used at a later stage to diagnose and correct the error. A permanent record of any information that will be helpful in this process should be kept.

Identify the error kernel

One of the basic elements of system design is identifying which part of the system has to be correct and which part of the system does not have to be correct.

In conventional operating system design the kernel of the system is assumed to be, and must be, correct, whereas all user application programs do not necessarily have to be correct. If a user application program fails this will only concern the application where the failure occurred but should not affect the integrity of the system as a whole.

The first part of the system design must be to identify that part of the system which must be correct; we call this the error kernel. Often the error kernel has some kind of real-time memory resident data base which stores the state of the hardware.

Processes, Servers and Messages

Processes

Implement a process in one module

Code for implementing a single process should be contained in one module. A process can call functions in any library routines but the code for the "top loop" of the process should be contained in a single module. The code for the top loop of a process should not be split into several modules - this would make the flow of control extremely difficult to understand. This does not mean that one should not make use of generic server libraries, these are for helping structuring the control flow.

Conversely, code for no more than one kind of process should be implemented in a single module. Modules containing code for several different processes can be extremely difficult to understand. The code for each individual process should be broken out into a separate module.

Use processes for structuring the system

Processes are the basic system structuring elements. But don't use processes and message passing when a function call can be used instead.

Registered processes

Registered processes should be registered with the same name as the module. This makes it easy to find the code for a process.

Only register processes that should live a long time.

Assign exactly one parallel process to each true concurrent activity in the system

When deciding whether to implement things using sequential or parallel processes then the structure implied by the intrinsic structure of the problem should be used. The main rule is:

"Use one parallel process to model each truly concurrent activity in the real world"

If there is a one-to-one mapping between the number of parallel processes and the number of truly parallel activities in the real world, the program will be easy to understand.

Each process should only have one "role"

Processes can have different roles in the system, for example in the client-server model.

As far as possible a process should only have one role, i.e. it can be a client or a server but should not combine these roles.

Other roles which process might have are:

Supervisor: watches other processes and restarts them if they fail. Worker: a normal work process (can have errors). Trusted Worker: not allowed to have errors.

Use the process dictionary with extreme care

Do not use get and put etc. unless you know exactly what you are doing! Use get and put etc., as little as possible.

A function that uses the process dictionary can be rewritten by introducing a new argument.

Don't program like this:

(defun tokenize
  ((`(,head . ,tail))
    ...)
  (('())
    (case (get-characters-from-device (get 'device))
      ('eof
        '())
      (`#(value ,chars)
        (tokenize chars)))))

The correct solution:

(defun tokenize
  ((device (,head . ,tail))
    ...)
  ((device '())
    (case (get-characters-from-device device)
      ('eof
        '())
      (`#(value, chars)
        (tokenize device chars)))))

The use of get and put will cause a function to behave differently when called with the same input at different occasions. This makes the code hard to read since it is non-deterministic. Debugging will be more complicated since a function using get and put is a function of not only of its argument, but also of the process dictionary. Many of the run time errors in LFE (for example bad_match) include the arguments to a function, but never the process dictionary.

Servers

Use generic functions for servers and protocol handlers wherever possible

In many circumstances it is a good idea to use generic server programs such as the generic server implemented in the standard libraries. Consistent use of a small set of generic servers will greatly simplify the total system structure.

The same is possible for most of the protocol handling software in the system.

Write tail-recursive servers

All servers must be tail-recursive, otherwise the server will consume memory until the system runs out of it.

Don't program like this:

(defun loop ()
  (receive
    (`#(msg1 ,msg1)
     ...
     (loop))
    ('stop 'true)
    (other
     (logger:error "Process ~w got unknown msg ~w~n"
                   `(,(self) ,other))
     (loop)))
  ;; don't do this! This is not tail-recursive!
  (io:format "Server going down" '()))

This is a correct solution:

(defun loop ()
  (receive
    (`#(msg1 ,msg1)
     ...
     (loop))
    ('stop
     (io:format "Server going down" '()))
    (other
     (logger:error "Process ~w got unknown msg ~w~n"
                   `(,(self) ,other))
     (loop))))

If you use some kind of server library, for example generic, you automatically avoid doing this mistake.

Messages

Tag messages

All messages should be tagged. This makes the order in the receive statement less important and the implementation of new messages easier.

Don't program like this:

(defun loop (state)
  (receive
    ...
    (`#(,mod ,funcs ,args)
     (erlang:apply mod funcs args)
     (loop state))
    ...))

The new message `#(get_status_info ,from ,option) will introduce a conflict if it is placed below the `#(,mod ,funcs ,args) message.

If messages are synchronous, the return message should be tagged with a new atom, describing the returned message. Example: if the incoming message is tagged get_status_info, the returned message could be tagged status_info. One reason for choosing different tags is to make debugging easier.

This is a good solution:

(defun loop (state)
  (receive
    ...
    (`#(execute ,mod ,funcs ,args)
     (erlang:apply mod funcs args)
     (loop state))
    (`#(get_status_info ,from ,option)
     (! from `#(status_info ,(get-status-info option state)))
     (loop state))
    ...))

Use tagged return values

Use tagged return values.

Don't program like this:

(defun keysearch
  ((key `(#(,key ,value) . ,tail))
    value)
  ((key `(cons `#(,wrong-key ,wrong-value) . ,tail))
    (keysearch key '()))
  ((key '())
    'false))

Then (tuple key, value) cannot contain the false value.

This is the correct solution:

(defun keysearch
  ((key `(#(,key ,value) . ,tail))
    `#(value ,value))
  ((key `(#(,wrong-key ,wrong-value) . ,tail))
    (keysearch key '()))
  ((key '())
    'false))

Flush unknown messages

Every server should have an Other alternative in at least one receive statement. This is to avoid filling up message queues. Example:

(defun main-loop ()
  (receive
    (`#(msg1 ,msg1)
     ...
     (main-loop))
    (`#(msg2 ,msg2)
     ...
     (main-loop))
    (other  ; flush the message queue
     (logger:error "Process ~w got unknown msg ~w~n" `(,(self) ,other))
     (main-loop))))

Interface functions

Use functions for interfaces whenever possible, avoid sending messages directly. Encapsulate message passing into interface functions. There are cases where you can't do this.

The message protocol is internal information and should be hidden to other modules.

Example of interface function:

(defmodulee fileserver
  (export
   (start 0)
   (stop 0)
   (open-file 1)
   ...))

(defun open-file (server-pid filename)
  (! serever-pid `#(open-file-request ,filename))
  (receive
    (`#(open-file-response ,result) result)))

...

Time-outs

Be careful when using after in receive statements. Make sure that you handle the case when the message arrives later.

Trapping exits

As few processes as possible should trap exit signals. Processes should either trap exits or they should not. It is usually very bad practice for a process to "toggle" trapping exits.

Software Components

From the smallest chunks of code to a completed project.

Flow Control

`if` Branches

Large conditional expressions and deeply nested blocks of code are harder to read, so should be factored out into functions.

For example, this:

(if (and (fuelled? rocket)
         (lists:all #'strapped-in?
                    (crew rocket))
         (sensors-working? rocket))
  (launch rocket)
  (! pid `#(err "Aborting launch.")))

Should be refactored to something like this:

(defun rocket-ready? (rocket)
  (and (fuelled? rocket)
       (lists:all #'strapped-in?
                  (crew rocket))
       (sensors-working? rocket)))

(if (rocket-ready-p rocket)
  (launch rocket)
  (! pid `#(err "Aborting launch.")))

`case` Branches

Don't write complex case statements with deeply nested branching. Instead, split these into functions, too, pattern-matching in the function heads.

Functions

Keep Functions Small

Keep functions small, focused on one thing. If you have six separate tasks being performed by a function, create six functions for these.

Group Functions Logically

Try to always separate unexported and exported functions in groups, with the exported ones first, unless it helps readability and code discovery.

Modules

When defining modules in LFE, put exported functions and their arities on separate lines. Optionally, functions of the same name with different arity may be put on the same line. Functions within a single export call should be sorted alphabetically.

Do not use (export all); explicitly exporting functions constitutes the conventional contract for clients utilising the module.

Very bad:

(defmodule maths
  (export all))

Bad:

(defmodule maths
  (export (factorial 2)
          (large-prime-number? 1)
          (small-prime-number? 1)
          (ackermann 2)
          (factorial 1)))

Better:

(defmodule maths
  (export
   (ackermann 2)
   (factorial 1) (factorial 2)
   (large-prime-number? 1)
   (small-prime-number? 1)))

If you have a public API with groups of related functions in a module, you may indicate their relatedness with separate exports:

(defmodule maths
  (export
   (util-func 1)
   (other-util 2))
  (export
   (ackermann 2)
   (factorial 1) (factorial 2)
   (large-prime-number? 1)
   (small-prime-number? 1)))

With Pseudo-Packages

If you are using the LFE rebar3 plugin, then you also have the flexibility of organising your project's source code into sub-directories under your source directory (see the projects section for more information).

In that case, you would define your module like so:

(defmodule project.subdir.maths
  (export
   (ackermann 2)
   (factorial 1) (factorial 2)
   (small-prime-number? 1)
   (large-prime-number? 1)))

Since there is no support in Erlang and LFE for actual packages, the dotted name is actually a module. As such, when referencing this module elsewhere, use import aliases to improve readability):

(defmodule client
  (export
   (some-func 0))
  (import
   (from project.subdir.maths
         (small-prime-number? 1))))

(defun some-func ()
  (small-prime-number? 100))

Or, if you need to avoid a name collision between the imported function and one in the client module:

(defmodule client
  (export
   (some-func 0))
  (import
   (rename project.subdir.maths
           ((small-prime-number? 1) small-prime?))))

(defun some-func ()
  (small-prime? 100))

When to Create

If some portion of your code is reusable enough to be a module then the maintenance gains are really worth the overhead of splitting it out with separate tests and docs.¹

Gains for separating code into separate modules include, but are not limited to:

Easier reuse in other parts of the software being developed.
Increased ability to reason about problems due to increased simplicity and separation of concerns.
Great clarity and understanding of the system as a whole.

A good general workflow around module creation:

Start small and remain focused on the problem at hand.
Write just the functions you need.
Keep the functions small and limit them to one specific chunk of functionality (do one thing and do it well).
Make incremental changes as needed.

For new code:

Experiment with in the LFE REPL by defining your function and then calling with different values (expected and otherwise).
When it works in the REPL, create a test module in ./test and paste the function calls in a test case.
Create a new module in ./src and paste the final form of your function from the REPL.
Ensure the tests pass successfully for the new module.

Build your libraries using this approach

Parts of this page were adapted from a Github Gist by James Halliday.

Libraries

Use Before You Write

Look for libraries that solve the problems you are trying to solve before embarking on a project.¹ Making a project with no dependencies is not some sort of virtue. It doesn’t aid portability and it doesn’t help when it comes to turning a Lisp program into an executable.

Writing a library that solves the same problem as another hurts consolidation. Generally, you should only do this if you are going to make it worth it: Good, complete documentation, examples, and a well-designed website are – taken together – a good reason to write an alternative library to an existing one.

As always, check the licensing information of the libraries you use for incompatibilities.

Writing Libraries

Before starting a project, think about its structure: Does every component have to be implemented within the project? Or are there parts that might be usable by others? If so, split the project into libraries.

If you set out to write a vast monolith and then lose interest, you will end up with 30% of an ambitious project, completely unusable to others because it’s bound to the rest of the unfinished code.

If you think of your project as a collection of independent libraries, bound together by a thin layer of domain-specific functionality, then if you lose interest in a project you will have left a short trail of useful libraries for others to use and build upon.

In short: write many small libraries.

Notes

¹ This entire page was adapted from the lisp-lang.org Style Guide's General Guidelines.

Projects

Use the established best practices for Erlang project creation, adapted for LFE.

These have been used when defining the LFE new project templates in the rebar3 plugin. That is probably the best way to get consistent results when creating the most common types of LFE projects (e.g., main-scripts, escripts, libraries, OTP applications, and OTP releases).

With Pseudo-Packages

While Erlang and LFE do not support packages, it is possible to use the rebar3 LFE plugin to simulate packages, complete with project directory structures that consolidate specific functionality in collections of sub-directories. These will be detected at compile-time when you use rebar3 lfe compile and from these, proper Erlang-compatible modules will be compiled (with dotted names preserving the hierarchy). Consider your project's organisation carefully when creating your sub-directory structure.

Here is a good example for a game project's directory structure:

 ├── LICENSE
 ├── README.md
 ├── rebar.config
 ├── src
 │  ├── title.app.src
 │  └── title
 │     ├── config.lfe
 │     ├── db.lfe
 │     ├── graphics
 │     │  ├── mesh.lfe
 │     │  ├── obj.lfe
 │     │  └── gl.lfe
 │     └── logging.lfe
 ├── test
 ...

The modules under the src/title directory can be represented in a more abstract hierarchy:

title
  title.graphics
    title.graphics.mesh
    title.graphics.obj
    title.graphics.gl
  title.config
  title.logging
  title.db

Keep in mind that the LFE plugin will flatten this structure into sibling files compiled to the ebin directory:

title.app
title.config.beam
title.db.beam
title.graphics.mesh.beam
title.graphics.obj.beam
title.graphics.gl.beam
title.logging.beam

Software Engineering

General best practices around software engineering.

Principles

There are some basic principles for team software development that every developer must keep in mind. Whenever the detailed guidelines are inadequate, confusing or contradictory, refer back to these principles for guidance:

Every developer's code must be easy for another developer to read, understand, and modify — even if the first developer isn't around to explain it. (This is the "hit by a truck" principle.)
Everybody's code should look the same. Ideally, there should be no way to look at lines of code and recognize it as "Alice's code" by its style.
Be precise.
Be concise.
Keep it simple.
Use the smallest hammer for the job.
Use common sense.
Keep related code together. Minimize the amount of jumping around someone has to do to understand an area of code.

Priorities

When making decisions about how to write a given piece of code, aim for the following in this priority order:

Usability by the customer
Debuggability/Testability
Readability/Comprehensibility
Extensibility/Modifiability
Efficiency (of the LFE code at runtime)

Most of these are obvious.

Usability by the customer means that the system has to do what the customer requires; it has to handle the customer's transaction volumes, uptime requirements; etc.

For the LFE efficiency point, given two options of equivalent complexity, pick the one that performs better. (This is often the same as the one that conses less, i.e. allocates less storage from the heap.)

Given two options where one is more complex than the other, pick the simpler option and revisit the decision only if profiling shows it to be a performance bottleneck.

However, avoid premature optimisation. Don't add complexity to speed up something that runs rarely, since in the long run, it matters less whether such code is fast.

Architecture

To build code that is robust and maintainable, it matters a lot how the code is divided into components, how these components communicate, how changes propagate as they evolve, and more importantly how the programmers who develop these components communicate as these components evolve.

If your work affects other groups, might be reusable across groups, adds new components, has an impact on other groups (including QA or Ops), or otherwise isn't purely local, you must write it up using at least a couple of paragraphs, and get a design approval from the other parties involved before starting to write code — or be ready to scratch what you have when they object.

If you don't know or don't care about these issues, ask someone who does.

Code of Conduct

The LFE Code of Conduct is the same as the erlang.org Code of Conduct, adopted from the original below for the LFE Community.

Overview

This Code of Conduct is a guideline of how to communicate within the LFE Community in a way we hope is easy to read, help mutual understanding and avoid flames. The LFE Community is by definition all communication in or around LFE use, development, and related discussions, including but not limited to the LFE mailing lists, Slack and Discord discussions, Github PRs and reviews, and even social media.

If you have been pointed to this page after posting a message, the reason is probably that you made some common mistake that we all made at least once. We have been there, do not feel alone, and please keep reading. It will help all of us in the LFE Community.

It is pointless to send a message that only warns about posting style. If you are trying to point someone to correct posting style guidelines, please do so while at least honestly attempting to answer their questions or comments. It is generally unhelpful to give only a warning related to posting style, as newcomers may feel unwelcome, only to leave. And that is exactly what we do not want.

Content Policy

Be nice to each other.
Hateful, hurtful, oppressive, or exclusionary remarks are not tolerated. (Cursing is strongly discouraged, but if you must curse, it may under no circumstances be used to target another user or in a hateful manner).
No SPAM.
The common language spoken in the LFE Community is English. The are various native language communities around the world. Conversely, many members of this list do not speak or write English as a first language. Be understanding of erroneous English, and don't point it out if it doesn't change the meaning or make it ambiguous.
The mailing lists, Github issues, and Github PRs are to discuss LFE, its uses, contributions, and its community. Try to keep discussions in those areas on topic.

Violating the Code

Posters of inappropriate content will be (1) informed, (2) warned if they continue breaking the rules, (3) moderated, and ultimately (4) banned to retain the highly welcoming and friendly nature of the LFE Community. To lift a ban or otherwise contend a regulative measure, please contact maintainers@lfe.io.

Part VIII - Conclusion

Feedback and Documentation Bugs

If you would like to provide feedback about this guide, we would welcome the chance to improve the experience for everyone. Please create a ticket in the Github issue tracker. Be sure to give a full description so that we can best help you!