Projects

Captura

This is my main obsession lately. This website runs using captura and includes the captura-text plugin - for more see the about page. I have been building captura as I desire the functionality in another application I have been planning to build for a while and would be frustrated to have to re-engineer this functionality.

What is Captura?

Captura is a 'framework' to build HTTP APIs using plugins. For a technical description please see captura on github. Essentially captura can be used on top of existing services to implement sharing and collections amongst users - it can be used as a 'front door' to services and provide users with the ability to sort api objects (called documents) into collections. Every captura object has a content section which a plugin can decide to specify using generics (in progress). One may think of something like 'google docs' which stores arbitrary objects - for instance references to resources defined by other services. documents can be shared at various levels, for instance a document owner may grant other users the permissions to own, modify, or view a document - additionally documents may be public or not.

Captura is easy to set up for experimentation, especially on linux machines that include docker. It is as easy cloning the project and running the docker compose project as shown in the instructions on github.

Captura is build upon FastAPI and therefore will provide autogenerated openapi.json data and documentation with redoc or swagger. To get started using captura or to request a demo go to captura.acederberg.io, to access documentation see captura.acederberg.io/docs or captura.acederberg.io/redoc.

Key Features

  • FastAPI Application - Starting the API is as easy as running captura run.
  • Typer Client - A client powered by httpx and typer. This gives a convenient means to reproduce test results and debug failed tests directly from the terminal and can be helpful for developers.
  • PyTest Tests - Fast and scalable (using pytest-xdist and the --count flag) integration and unit tests.
  • Dummies - A means to generate dummy data for the MySQL database. I found this to important in writing tests in a way that does not involve writing out manifests for data, which in the past I have found to be very inefficient. This also makes it easy to find flakey tests since tests do no recieve the same dummy data every time.
  • Configurability - All of the above are easy to configure using YAML or JSON as the result of another library I've built and maintained, see yaml-settings-pydantic. Futher, tests are configurable to run many times and there are additional settings that may be included in pyproject.toml.
  • Plugins - Captura is easy to extend for most purposes. One can easily extend the fastapi application or typer client. For an example of a plugin see captura-text.
  • Docker - The docker-compose project makes it very easy to get started for experimentation and contribution.

Captura Deploy

For those who are interested in accessing tools used to deploy captura to this domain, please let me know via email. For the time being, the repository is private until I can gaurentee that my infrastructure is relatively secure.

Key Features and Tools

  • Pulumi - Infrastructure as code (IaC) tool used to provision resources in linode and the resulting kubernetes cluster. I chose this my experience with domain specific languages (for instance HashicorpLang, which is used by terraform and similar tools) has been relatively inconvenient.
  • Kubernetes - Managed by pulumi. This includes traefik, captura, error-pages, and a registry.
  • Automatic DNS - When pulumi provisions a node balancer for kubernetes, it is possible to point a domain name to that DNS using the porkbun HTTP API.
  • Building of Docker Images - Docker images are build in a standardized way. In the past, I found it inconvenient to have many docker build scripts that had to be maintained accross multiple pipelines. This resulted in a lack of uniformity in naming and tagging of docker images, the need to provide secrets to multiple pipelines, and the dispersion of such tasks accross many repositories - hence why universal builds are necessary.

Yaml Settings Pydantic

This is a library I made, maitain, and use which has some use by others beyond myself - this is also a dependency of captura.

I made this because I often use YAML data since other configuration formats are easier to write incorrectly, harder to read, and are generally not as powerful as YAML. Futher, YAML is the data format that I use most often since some of my favorite tools (kubernetes, pulumi, and various pipelines or actions like tools) use this format.

For more please see yaml-settings-pydantic.

This Website

Unfortunately for backend and devops engineers, the truth is that seeing is believing. Such has been my experience in my work - executives and customers usually do not care about back-end optimizations and implementation details, and instead only have interest in what can be put before them with no effort, such as a browser/mobile application.

One might compare this to constructing an engine - few may take interest in the engine until it is dropped into, for instance, a racecar - and even then, depending on the audience of course, few may take interest in the engine itself despite the fact that there would be no racecar without it. To myself it makes sense that few would desire to venture beyond the scope of thier direct experience as digging deeper requires effort, and said effort may not come to any useful fruition besides the satisfaction of their intellectual curiousity.

Since I am a ways away from having a nice front-end for captura, I wanted to provide a few things on this site for anybody here to look at. For this, please see the about page.