I run a small tower server in my house to provide services that I use for software development, recently CentOS-8-Stream reached end of life. Since this server distribution doesn’t support in-place updates, updating requires reinstalling with a new version of the CentOS-9-Stream. This post documents the challenges that I hit along the way.
As an academic, you write a lot of papers. If you contribute to the disciplines of Math, Computer Science, or some domains of physical sciences, you’ve probably used LaTeX – a typesetting language – through a tool like Overleaf. What’s nice about LaTeX is that it attempts to separate concerns: Authors with minimal effort can port the text and structure of a document from one style to another. It generally produces much more ascetically pleasing output by default than what most authors would do on their own. It handles formatting of three things that generally are extremely frustrating in other tools: citations/bibliography, cross-references, footnotes/endnotes, equations, and embedded code listings. When used with a source control like Git it provides one of the best change review and tracking mechanisms of any document platform.
CI/CD is a critical, but difficult to get right part of software engineering. You often want to test multiple distributions, multiple compilers on each commit, and you want that to be as fast as reasonably possible. This gets more complicated when you have large dependency trees that you want to remain consistent. Recently, I adapted the CI/CD system for a project that I maintain LibPressio to use Dagger – a programmatic way to do CI/CD portable-ally across runner environments which made it easier to run our tests and verify correctness.
Most successful langauges have a “killer” features that motivates
In my previous post, I commented on the tools that I use for software development, but I didn’t talk about either the process of choosing a language or the libraries within a language that I use most frequently. This post expands on how I work within a language and specific “killer” libraries that I use most and how they compare to facililties that I know from other languages.
From time to time I get a question of what tools I use for what jobs. Here the tools I use to get things done:
Clang style refactoring has been something I’ve been admiring from afar for
quite a while now. However for a user to actually use it, it has previously
required forking llvm to be able to use it in a reasonable fashion because of
things like the hack in clang used to locate the resource directory or other
fragile hacks like LD_PRELOAD.
Have you ever wanted to identify a list of files that would need modifications to adopt a new API? Clang-Query can make this much easier. I recently wanted to introduce a set of helper functions to simplfy an aspect of compressor configuation in LibPressio. But first, I needed to know what modules were effected.
In the past, I’ve written pretty glowingly about Julia. It’s been a few years since I first used Julia in 2019, and it hasn’t completely replaced Python for me. However, I wanted to share a few neat projects that I’ve done using it which would have been much more painful without it, and share what I think now about what I wrote in 2019.
High-performance network interconnects, such as Infiniband, are common in high performance computing environments. Recently, my colleagues, and I ran a series of experiments on the Cooley system at the Argonne Leadership Computing Facility. For one set of experiments, the network performance was consistent and fast, but for the others we periodically had poor performance. Up until this point in my work, I had not directly used or configured network libraries below this layer. This post summarizes the steps we took to investigate the performance.
Modern software does not exist in a vacuum, developers always rely on other developers or engineers to provide aspects of their systems. Even if you write hello world in your programming language of your choice you likely still depend on the standard library from your language. Even if you were to write hello world in the raw binary format of your assembly language, you still depend on the operating system. Even if you wrote your own uni-kernel to produce hello world on the console, you likely rely on vendor provided firmware to initialize the program or memory. And if you are one of the blessed few who can completely fabricate your own hardware and don’t need to rely on vendor provided firmware, you likely don’t have time or expertise to write efficient higher level systems and need others to provide software to run on your hardware. Therefore the question is not will you use dependencies, but how will you use them responsibly.