Using Emacs Org-mode As My Package Manager

I use Org-mode mainly for note-taking and blogging, instead of literal programming. So I actually started with a plain org-table, like this:

| Package | Type   | Reason                         |
|---------+--------+--------------------------------|
| figlet  | (app)  | Print big letters in terminals |
| toilet  | (app#) | See ~figlet~                   |

I use # to mark packages that I am trying out, which I will remove if they don’t leave an impression on me or existing packages are satisfactory enough. Once in a while, when I run pacman -Syu to update things, I would glimpse the list of packages, run pacman -Qi on those that are unknown to me, and add explicitly installed ones to this table.

Soon, this table grew to several scroll-up-command in Emacs, and I thought: let me try to take note of every explicitly installed package on my system so that I understand what EndeavourOS and I have actually installed.

And this is when I started putting all kinds of Org-mode babel blocks into that note file.

To put all those packages into the table, as an Org-mode user, naturally I decided to write a Org-babel script for that, since I need to do it every time things are out-of-sync. In Org-mode, you start by naming the table with #+name: installed to enable using the table as a data source for your Org-babel scripts:

#+name: installed-groups
| Package     | Type   | Reason |
|-------------+--------+--------|
| linux-tools | (base) |        |
| texlive     | (app)  |        |
| fcitx5-im   | (cjk)  | IME    |

#+name: installed
| Package    | Type   | Reason        |
|------------+--------+---------------|
| base-devel | (base) | You need this |

Then you might use the data as you wish:

#+begin_src elisp :var packages=installed :var groups=installed-groups :results table
  (setq upstream-installed (shell-command-to-string "pacman -Qeq"))
  (setq installed-from-groups
        (cl-loop for g in groups concat (shell-command-to-string (concat "pacman -Qgq " g))))
  ;; compute: new = upstream-install sans installed sans installed-from-groups
  ;; compute: rem = installed plus installed-from-groups sans upstream-install
  ;; print as another table
  (append '(("" "Potentially Removed Packages:"))
          rem
          '(("" "Newly Installed Packages:"))
          new)
#+end_src

(let* ((with-reasons (mapcar #'car packages))
       (groups (mapcar #'car group-table))
       (with-reasons-set (make-hash-table :size (length with-reasons) :test 'equal))
       (upstream (split-string (shell-command-to-string "pacman -Qeq")))
       (upstream-set (make-hash-table :size (length upstream) :test 'equal)))
  (dolist (p with-reasons) (puthash p t with-reasons-set))
  (setq mine-tmp-upstream (seq-filter (lambda (p) (not (gethash p with-reasons-set))) upstream))
  (dolist (p upstream) (puthash p t upstream-set))
  (dolist (g groups)
    (let ((gpackages (split-string (shell-command-to-string (concat "pacman -Qgq " g)))))
      (dolist (p gpackages)
        (if (gethash p upstream-set)
            (remhash p upstream-set)
          (puthash p t with-reasons-set)))))
  (append '(("" "Potentially Removed Packages:"))
          (mapcar (lambda (p) (list p "" "" ""))
                  (seq-filter (lambda (p) (not (gethash p upstream-set))) with-reasons))
          '(("" "Newly Installed Packages:"))
          (mapcar (lambda (p) (list p "" "" ""))
                  (seq-filter (lambda (p) (not (gethash p with-reasons-set)))
                              (seq-filter (lambda (p) (gethash p upstream-set)) upstream)))))

Then, press C-c C-c with the cursor in the code block, and Org babel will execute the code for you, producing a new table with all the out-of-sync entries, like this:

#+begin_src elisp :var packages=installed :var groups=installed-groups :results table
  ;; diff
#+end_src

#+RESULTS:
|               | Potentially Removed Packages: |
| jdk21-openjdk |                               |
|               | Newly Installed Packages:     |
| hotspot       |                               |

Then, once in a while I would look at this new table, find something that looks interesting to me, research it, and put it into the installed table. After a long while, I finally got every explicitly installed package covered.

Having to sync by C-c C-c after (un)installing packages seems annoying, so I went further ahead and added a new babel block for installing packages from the Orgmode note file.

Actually, let’s add a block for searching for packages first. We’ll read all packages from pacman, and use completing-reading to prompt the user (that is, me) for the package name:

(let* ((upstream (split-string (shell-command-to-string "pacman -Slq")))
       (pkg (completing-read "Package: " upstream)))
  (term (concat "pacman -Si " pkg)))

It’s a bit slow as it runs pacman -Slq to get the package list. But personally, the auto-complete experience is actually better than my personal Zsh config, where every tab-completion also seems to fetch the package list afresh.

Also, if we are to always use Org-mode for package management, we can very easily add a cache for this. But I’m just too lazy for that.

Similarly, when installing a package from Org-mode, we prompt for the package name (or package names, with completing-read-multiple), use org-link-open-from-string to jump to our installed table, update it, jump back, and emit an extra babel block for the user to execute, which actually installs things with pacman.

…Confused? That means, when you C-c C-c on the following babel block:

#+begin_src elisp :var packages=installed :wrap src bash
  (setq upstream (fetch-all-packages))
  (setq package (completing-read "What to install: " upstream))
  (update-our-installed-table package (read-string "Reason: "))
  (prin1-to-string (list 'term (format "sudo pacman -S %s" package)))
#+end_src

It will update things in the note file, and then generate a new babel block (thanks to the :wrap src elisp header argument), like this:

#+RESULTS:
#+begin_src elisp
  (term "sudo pacman -S linux")
#+end_src

…which you can use C-c C-c again to confirm and execute, to actually install that package (or use C-/ to undo the table change).

(let* ((upstream (split-string (shell-command-to-string "pacman -Slq")))
       (pkg-string (completing-read-multiple "Package(s) to install: " upstream))
       (pkgs (read (concat "(" (string-join pkg-string " ") ")")))
       (reasons (seq-filter
                 (lambda (s) (not (string-match-p "#" s)))
                 (seq-uniq (mapcar #'cadr (cdr packages)))))
       (reason (completing-read "Category: " reasons))
       (reason (if (yes-or-no-p "Temporary? ")
                   (concat (string-trim-right reason ")") "#)")
                 reason))
       (note (read-string "Notes for the package(s): ")))
  (save-excursion
    (org-link-open-from-string "[[installed]]")
    (forward-line)
    (goto-char (org-table-end))
    (insert
     (mapconcat
      (lambda (pkg) (format "| %s | %s | %s | |\n" pkg reason note))
      pkgs))
    (org-table-align))
  (prin1-to-string (list 'term (format "sudo pacman -S %s" (mapconcat #'symbol-name pkgs " ")))))

Of course we also need another babel block for uninstalling things, which will prompt the user to select a package from the installed table, optionally move the info to an uninstalled table for track record, and then emit a different uninstalling babel block. But otherwise it is quite similar to installing packages.

(let* ((packages (if (yes-or-no-p "Temporary? ")
                     (seq-filter (lambda (pkg) (string-match-p "#" (cadr pkg))) packages)
                   packages))
       (pkgs (mapcar #'car packages))
       (pkg (completing-read "Package to remove: " pkgs)))
  (save-excursion
    (org-link-open-from-string "[[installed]]")
    (re-search-forward (format "^|[ \t]*%s[ \t]*|" (regexp-quote pkg)))
    (let ((line (buffer-substring (pos-bol) (pos-bol 2))))
      (save-excursion
        (org-link-open-from-string "[[uninstalled]]")
        (forward-line)
        (goto-char (org-table-end))
        (insert line)
        (org-table-align))
      (delete-line)))
  (prin1-to-string (list 'term (format "sudo pacman -Rs %s" pkg))))