Metaist

ds 1.3.0

Run commands without activating virtual environments; support for Makefile and more.

ds 1.3.0 is available. This release represents a shift from only supporting the overlap of all file formats to specific parsers for each supported format.

To install ds:

python -m pip install ds-run

# or, if you use uv:
uv tool install ds-run

# or, install the standalone version:
wget -O ~/.local/bin/ds -N https://github.com/metaist/ds/releases/latest/download/ds
chmod +x ~/.local/bin/ds

Read more.

Combined Release Notes 1.1.0, 1.2.0, 1.3.0 - 2024-08-29T13:08:58Z #

Unlike the official Changelog, these notes are organized by feature.

Documentation #

I got some feedback from Shalev NessAiver that many HackerNews comments often complain that it takes too long to see real examples, so I tried to illustrate a bunch of salient examples up top.

I often strive to get 100% branch coverage in unit tests, so if I disable branch coverage, I want to remember why (e.g., hard to model catching CTRL+C).

Logging #

There are three kinds of logging I use:

  1. Tracing execution flow: Where did execution reach?
  2. Reporting changes of state: What is the state of the system?
  3. Providing information content: What useful things should you know?

One thing I think is interesting is providing some guidance (usually behind a --debug flag) is providing information about how to enable or disable particular behavior. This is particularly helpful when you wonder about why the program is behaving a certain way.

Install #

uv is emerging as the dominant alternative to pip and all the other python project management tools.

Shalev NessAiver helped me figure out the two things I needed to build a truly cross-platform (Linux, macOS, Windows) binary executable (using Cosmopolitan), which is actually just a very carefully crafted zip file:

  1. The special zip commands to add files to the packaged python.
  2. How to add arguments so that ds runs instead of the python REPL.

With this release ds can be installed in three ways: via pip, as a uv tool, and as a standalone binary.

Read more about packaging python projects with Cosmopolitan.

Task Description #

help is inspired by pdm; adding the command wrapping made it somewhat nicer to look at the output of ds as it's running. I wrote a related post on figuring out how to do wrap bash commands.

Task Environment #

Moving env_file loading later lets you generate that file in some task and letting another task read that file.

The purpose of this feature is to let you use a config file in one directory and executes the tasks in another directory (i.e. not the directory containing the config file).

Task Formats #

Joe Hostyk convinced me to support a minimal subset of the Makefile format. Many people use make as a way to alias long commands with a short name. I implemented the aliasing and composites and a few other features when they were relatively easy to implement.

While I was implementing the Makefile format, I realized that while I allow both composite and cmd/shell properties on tasks when running the task, I didn't allow them in the parser. So I fixed that to more clearly communicate the intent that composite represents perquisites that run before cmd/shell.

This style turned out to be helpful for formats like Makefile which use $@ to mean something else.

Inspired by Cargo and easy to support.

poetry only really supports a single version of the python call format. Previously, I was was against supporting call-type tasks because they are language-specific and I had a universal parser for all the file formats. However, once I implemented #77 and every file format got its own parser, I could reintroduce the concept of language-specific features, and so I added poetry back in.

Task Runner #

Several of the formats support some kind of lifecycle event. I was reluctant to add support for these because they obscure the flow of execution. However, I ended up deciding that I could have people explicitly opt-in to running additional tasks with command-line options. See my separate post on how I changed my mind about lifecycle events.

Unlike --list, --dry-run will show you the actual command that is about to run (will full argument interpolation, etc.).

I really didn't like pnpm's solution to use a regex to specify which tasks to run and using globs has felt very natural.

Originally, I thought this was a weird thing to allow and had a special flag to disable it. But now with #73 and #74, I routinely just want to execute a command in the context of the project (e.g., ds 'echo $PATH').

Detecting the SHELL turned out to be surprisingly easy for POSIX machines and hilariously difficult on Windows. I've kept the Windows code as "EXPERIMENTAL" because I don't have a good way of testing it.

Shalev NessAiver provided some feedback to the 1.2.0 release suggesting that the project-specific folders should be on the PATH without having to activate or otherwise specify them. Currently only project folders for node and python are supported, but this could easily extend to php. One question that is still somewhat open is under which conditions the searches should take place. I'd also like a way to move the logic of the search into the respective file formats, but I don't have a clean way to do that yet.

This was the most significant bit of work for this release that barely added any new features. The main goal was to try and parse each file format as strictly as possible while supporting the general goal. A few features that got added were call for pdm/rye/composer and argument sharing from pdm. I wrote a post about how I changed my mind about call-style tasks.

During work on #77, I cleaned up the search order and removed the never-used .ds.toml file.

While reading bits of the uv codebase, I saw this naming convention and decided to switch to it.

Testing

This became the topic of my most-liked tweet. The point isn't to run this all the time, but to simulate what GitHub Actions does when running all my tests on every supported version of python.

Just some clean up.