The best tools are the tools you will use

Over the years I have bought many devices and tools, admittedly I’m always interested in new tools that can help me do more with my limited time as a professional with a fulltime job.

I have owned laptops, tablets, desktop computer, drawing tablet, gaming consoles, handheld consoles,… But what I found was that the best tool for the work I do is not always the highest spec’d devices, but the one that is the most convenient.

When it comes to digital painting, the iPad with Apple Pencil is great, drawing is smooth, battery life lasts whole day. A desktop PC with a screen tablet is also excellent due to no restrictions with thermal and energy.

I think that a gaming laptop is also a very great option for painting and development, especially game development, as it is both powerful and (somewhat) portable.

However, the tool that helped me get the most work done is this little thing: The Microsoft Surface Pro.

It’s not really that great

  • The screen is small.
  • It gets hot because it’s thin, but the processor produces lots of heat.
  • The keyboard is small.
  • It becomes slow when it’s hot.
  • Bugs.

But the one thing that make it beat everything else: it is very portable.

I can bring it anywhere, I can take notes on it using the pen, draw arts for my game, does programming, study, everything in one small device. And because I switched to the Surface Pro in 2013, my drawing skills improved tremendously because I drew A LOT more with it. I also did a lot of programming and development on it.

I brought it with me and drew and did programming on trains, on buses, on airplanes, at airports, at cafes, in bed,… everywhere.

Even though it sucks in almost everything I use it for, because of the convenience factor, I use it the most. If I have to wait until I come home to work on my projects on my desktop PCs, I would get much less done. The gaming laptop is too big and bulky to bring everywhere with me.

There same story applies to the Nintendo Switch.

BATMAN Arkham Knight Switch/PS4 Graphics Comparison #batman #arkhamknight #switch #ps4

The Switch plays most games worse than on other devices, it’s weak, small, blurry. But because it can suspend and resume games instantly and is small and portable, I ended up playing it much more.

Normal people may think that you should use an iPad to paint, a MacBook Pro to code, a beast of a gaming PC/consoles to game.

It makes me think, in many cases, the best tool for the job is not always necessarily the most high end tool. The most important thing in life for us is time, and convenience and accessibility will oftentimes help us do much more than fancier tools. Realizing this helps put ourselves in a more practical mindset, to be more critical of marketing and to choose the most practical tools that give more value to our lives.

The same applies to coding, software tools, sometimes the tools that will give you most value is the tool that you will use most, it may be something that is the most accessible, it may be an editor that is much faster to run, it may be something very unconventional. The best programming language could be the one that you are most familiar with to get the job done in a limited time budget, instead of listening to the crowd and choose what is trendy.

It’s good to explore alternatives to status quo and experiment to get reasonable outcomes.

[Dev log – DotnetGTD] Rewriting my task management app

I toyed with this idea for a while, my old task management app that I wrote in a few days had some limitations

  • It sends the whole database in JSON format every time something is changed.
  • The whole database is stored in memory of the browser, all the time.
  • There was no automated tests, and implementing new features was very difficult and easy to break things.
  • As a bonus, it is difficult to write tests for the app, since it was not designed to be testable from the start.

Because of those design decisions, there are a lot of consequences.

  • It’s hard to write tests to prevent regression.
  • That leads to it being hard to add new features or fix existing bugs without breaking something.
  • The whole database being stored in memory and sent to server every time something changes does not scale well with many tasks and projects (like 100s of thousands).

So I finally decided to rewrite the whole app using

  • Vite React for frontend.
  • Tailwind CSS for UI and UX development.
  • Vitest for frontend unit tests.
  • ASP.NET for REST WebAPI backend.
  • Entity Framework Core for Database ORM.
  • Azure SQL Server for database.
  • xUnit framework for backend unit tests.
  • Github Actions for CI/CD pipelines.
  • Docker for deployment.
  • Personal home server for hosting.
  • Cloudflare tunnels to give access over the Internet.

The project has been worked on for 5 months since June, it’s fully Test Driven Development and make use of the knowledge and experiences I have acquired from work and books.

This is actually the third time I have written a task management project, the first version was written back in 2018 and no longer compiles (you can read about it here).

The second version was written 2 years ago in 2023 and had all those limitations (there is a blog post here)

I decided that this version would be the last, as it is built as well as possible for easy maintenance and feature implementation. My experience in the past 5 months has been great. Finally the project is ready to use and I have migrated all my data from my previous task manager to the new one.

From this

To this

Yes, I use it to manage development of itself.

So far it is missing some small features from the previous version, but the main big features are there and ready to use to make me productive. I will add back some of the quality-of-life features back over time.

Rewriting the app is also a very good chance to apply all the knowledge I have learned from work and books, to add it to my portfolio and at the same time add value to both my professional and personal life.

I have found that it’s much more productive to focus on one project at a time to get one done and move on to the next, rather than trying to do a few projects at the same time and make tiny progress at each one. Now that I’m done with this, I may move on to the next project, it might be another full-stack application, a mobile app, or another video game project.

[Dev log – Expense manager] Using Tailwind CSS

Today I decided to try using TailwindCSS in my pet project, as I heard about it frequently and it’s also something rather different from other ways to implement UI styles.

With the Expenses Manager basic functionality done with tests and all, I added some styling to the UI using bootstrap so it’s a bit easier on the eyes. I created another branch to experiment with Tailwind instead and it looks interesting, so I decided to switch to use Tailwind for UI styles.

The UI with bootstrap

The UI with Tailwind

They look more or less similar, but the way the styles are applied are different.

With Tailwind, instead of creating CSS classes for different UI elements, you just use a bunch of pre-built CSS utility classes from Tailwind and adding those to the HTML element, you don’t write any CSS at all.

Instead each Tailwind class will modify the element to your liking, both layout and styles can be applied just by adding utility classes, which results a list of very long classes like following for just a button.

<button type="submit" 
   class="block rounded-md px-3.5 py-2.5 text-center 
         text-sm font-semibold shadow-xs 
         focus-visible:outline-2 focus-visible:outline-offset-2
         focus-visible:outline-indigo-600 w-full 
         bg-indigo-600 hover:bg-indigo-500 text-white">
   Login
</button>

For the above code for the Login button, the following styles were added

  • block: Display the button as a block.
  • rounded-md: Make the button border round.
  • px-3.5: Pad both sides so the horizontal padding is 3.5rem.
  • py-2.5: Pad both sides so the vertical padding is 2.5rem.
  • text-center
  • text-sm
  • font-semibold
  • shadow-xs
  • bg-indigo-600: Set background to indigo color.
  • hover:bg-indigo-500: Set background to lighter indigo color when hover over button.
  • text-white

The pros of Tailwind are that everything is self contained in the HTML code, so you don’t have to read the CSS classes or debug CSS in the browser to figure out how the styles are applied to elements. You can

  • Copy the code between projects, since the code is portable.
  • No need to write and maintain custom CSS.
  • The code is readable, you can basically read the classes to infer how they affect the element.
  • Modifying an element is safe as changing the classes does not affect other elements.

Tailwind works by scanning the source code, check for anything that looks like Tailwind classes, then generate a CSS files with only the styles that are needed by the project.

The downside obviously is that the code for a simple form looks very cluttered, as you can see from the example of a simple button. But reading around and also checking from their documentation, it’s obvious that Tailwind was designed to work with React components in mind.

For the above example, to reduce clutter, we simply wrap the button in a React Button component for reuse.

import { MouseEventHandler } from "react"

interface ButtonProps {
    text: string
    onClick: MouseEventHandler<HTMLButtonElement>
    extraClasses?: string
    mode?: ButtonMode
}

export enum ButtonMode {
    SECONDARY
}

export function Button({text, mode, onClick, extraClasses}: ButtonProps) {
    let classes = "block rounded-md px-3.5 py-2.5 text-center text-sm font-semibold " + "shadow-xs focus-visible:outline-2 focus-visible:outline-offset-2 " + ` focus-visible:outline-indigo-600 ${extraClasses} `
    switch(mode) {
        case ButtonMode.SECONDARY: 
            classes += " bg-gray-300 hover:bg-gray-200 text-black"
            break;
        default:
            classes += " bg-indigo-600 hover:bg-indigo-500 text-white"
    }
    return <button type="submit" 
                onClick={onClick}
                className={classes}>
                    {text}
                </button>
}

Once you have wrapped the UI control in its own Component, the Tailwind classes only need to be written once for that component, then you simply reuse your component many times without having to write Tailwind classes multiple times. The code is not cluttered at all.

<div>
  <Button text="Button A" onClick={e => alert("A clicked")} />
  <Button text="Button B" mode={ButtonMode.SECONDARY} 
    onClick={e => alert("B clicked")} />
</div>

So far, it’s very promising and interesting as well. It does seem like a good idea, an abstraction layer over CSS to make life easier for devs.

However, I had to update my Vitest tests because they were asserting some CSS classes on the UI. It would be a good idea to figure out a way to write tests that don’t change when the CSS framework is changed.

[Dev log – ReactGTD] Home style Continuous Delivery

Today I added a new feature to my React based GTD app. It was basically so that the web app remembers the last task view I was browsing so when I reopen it next time, it automatically opens the last view.

So for example, if I was last opening the Study view, the app would remember that, the next time I visit the app’s URL, it will automatically switch to Study view.

I wrote the whole thing using TDD and never ran the app once, and surprisingly after automatic deployment, it works the first time <3. Automation is beautiful.

I would like to blog a bit on my home style continuous delivery. Basically the app is hosted in a cheap second hand NUC mentioned in another blog post.

The server

The server

Once the code is pushed to a remote branch on Github, a PR is created and a Github workflow runs and execute the tests.

Then when the PR is merged to main, another Github workflow runs. The workflow builds the docker image and push to DockerHub registry.

Now the home style Continuous Delivery part: the home server has a cron job that runs every hour, the cronjob changes to the directory of the docker-compose file of the app, and then simply run

docker compose pull

docker compose up -d

It works, and it’s stupid simple xD. I remember the first time I wrote the app, I built and deployed the image manually many times, sometimes incorrectly because of some accidental changes to the files in the source code.

Now I just merge the pull request and wait until the app updates itself from the server.

[Dev log – Expense manager] Mocking and unit testing for Expense manager

Recently I have decided to write a new expense tracking application to replace the previous quick and dirty one I wrote long ago that I have been using to log my expenses.

The previous version was written using PHP and MySQL and was deployed using Docker on my personal home server. I got lazy to update it with new features as it was quite tedious, so I decided to write a new one using new technologies, also as a way to increase my portfolio of software products.

It is also a way to practice and make use of the new knowledge I acquired from reading books like Unit Testing, Clean Code, Test Driven Development and Refactoring. I also try to use as much knowledge I acquired from working in Optimizely.

The new expense tracking program is called “DotNetExpenseManager”, and will be developed with the following in mind:

  • Hosted on Github at https://github.com/brucengdev/DotNetExpenseManager .
  • The backend is written using C# and ASP.NET, it serves as a REST API backend for the frontend. xUnit is used for automated tests.
    • The backend uses Entity Framework and SQL server to store user data. When deployed on Linux or Docker, I’m using an Azure SQL Server Docker container.
    • Database migrations and updates are completely handled using EF Core migration by code.
  • The frontend is written using Vite and React in Typescript, using Vitest for automated tests with React-testing-library.
  • Fully TDD (Test Driven Development), for most of the time tests are written first before new code is written.
  • Apply knowledge I learned from the book Unit Testing principles.
  • The project must have Continuous Integration and Continuous Delivery implemented using Github actions and Github workflows.
    • Deployment is easy by using Docker and a docker-compose file. Deployment must be automated, so that I don’t spend much time doing deployment. I can just merge new code to the main branch, and everything is automatically deployed.

After few days of working on it, I managed to get the following functions working

  • Login/logout as an admin user.
  • Log a new expense entry

The UI is super nice, that means it looks absolutely barren 😀 . Because at the moment I’m focusing on implementing the functions first before implementing the UI.

Once login we have a very barren view of your current expenses for today 😀 .

You can log a new entry, but you can’t view your entries, because it has not been implemented xD .

For testing, the ideas are from the book Unit Testing Principles, Practices, and Patterns (Unit Testing Principles, Practices, and Patterns). The idea is

  • Mock the input and the output of the system, make sure that the implementation of those layers that glue external systems with the functional core is as straight forward as possible, so that they require little to no testing.
  • Try to isolate the functional part of the system as much as possible.
    • Test the functional core of the application as much as possible.

So for frontend, the localStorage and http client are mocked using real doubles. The doubles are real Javascript objects that run fast.

When the app runs, real implementations are used that uses Local Storage for IStorage and fetch APIs for http requests. This implementation makes it easier to change the underlying implementation easily.

So for example, in the future, the storage may not use localstorage, but using other ways of persistence, or you can totally switch to use other kinds of API for IClient (instead of REST, use Grpc, or GraphQL).

In the backend side, a similar design is used. The main logic (the functional core) is isolated into manager classes like AccountManager and EntryManager. The interface for clients which are the REST controllers is a thin layer that does only trivial tasks.

Similarly, the persistence layer which is the database are encapsulated into repositories like IUserRepository and IEntryRepository. The real implementation uses EntityFramework Database context, but these details are hidden away from the functional core (the managers). The repositories are created to be as straight forward as possible, so that testing it is trivial or not needed at all.

I have seen the practice to start a database for running tests, but I feel it’s still very slow. And it makes sure that your tests assume you will always use SQL server as your data source, when the database is something related to the infrastructure and it is not part of the business logic.

The benefits for designing this way: you can completely replace the backend data source with something else (MongoDB, Elasticsearch, another API). You can also add new API interface instead of just REST controllers, what about GraphQL?

This only works if we implement so that

  • The API interface is a very thin layer, the REST controllers (or GraphQL resolvers) do almost no business logic at all.
  • The persistence layer (EntryRepository) should do no business logic aside from simple CRUD (Create, Read, Update, Delete).
  • We put as much business logic as possible into the functional core (EntryManager) and cover it with tests.

In practice, I know that many projects rarely replace their database or API interface. But it’s fun nevertheless to try to implement this design.

With TDD, it’s pretty fun to work on, we almost never have to run the application manually to test its feature as the automated tests run very fast and can quickly verify that the software works. In many cases, I can make sure the tests pass, and the application runs correctly the first time I run it.

This is really helpful for a solo developer working on side projects like me, as the computer can do all the testing for me 😀 .

[Dev log] Getting Things Done (GTD) app with React and Redux

During some public holidays, I decided to rewrite BruceTodo – a GTD/Task management application I wrote in 2019, because BTD was written using multiple libraries that have had drastic changes since 2019, I couldn’t even build the old project anymore.

BTD was written as a way for me to learn React and Redux, I was new to the uni directional data flow design of React back then, so the code has some design decisions that make it less robust and more buggy.

I have not been very satisfied with many of the task management apps in the market as they are not tailored exactly to my needs, and work and life has been very chaotic with many things to keep track of. Some of the apps can provide 70% of the features, the rest requires more manual work to maintain.

Rewriting BTD will be useful to me to manage my work and life, it’s created to fit exactly my needs and I can also enhance it as needed.

Some decisions were made to improve the app and simplify maintenance:

  1. Instead of using a UI libraries/framework, use only plain HTML and Bootstrap.
    • Antd was changed drastically in later version, so it would be really troublesome for me to maintain and keep updating the app so it does not become unusable.
  2. Data are stored in a single JSON document instead of few tables in relational DB.
    • This drastically reduces the amount of backend code, and makes data updates much much simpler.
    • Data backup is also a matter of copying the JSON document from the server and store somewhere.
  3. The application is containerized in a single container with a volume to persist user data, similar to the cooking plan project.
    • This allows quick deployment by just pulling the latest Docker image and recreate the container.

The UI is very similar to the old one, it is designed for me so it’s most efficient and effective for me to allow me to quickly organize my tasks and projects. It’s for power users so no need to be super user friendly.

You have three separate views for tasks, projects and tags.

You can easily filter projects and tasks by the completion status, or whether the tasks and projects are current or to be done later. Tasks can also be filtered based on tags or projects. Since it’s React and everything is loaded into memory, filtering is super fast.

It’s also just a single click to mark a task as completed, or convert it to later or current task, you can also quickly set the project the task belongs to.

Each task can have multiple tags, so if a task can be done both at work and at home, you can add both tags to the same tag, and filter by the appropriate tag based on your current context.

Both projects and tasks can be quickly marked as Later, or make current by unchecking later. This allows me to quickly reorganize my current work depending on prioritization and workload.

The application has been deployed for personal use and so far it’s great, I will blog more about it in future posts on technical aspects.

Dev log [DotNetCookingPlan]: Deployment to personal server with Docker and Cloudflare tunnel

Today I worked on this little project for one hour and the main goal is to make it relatively easy to deploy the Cooking Plan application to my personal server for usage easily. For the first iteration I decided to use a simple method with Docker Compose. The goal is to make deployment to the server relatively easy so that changes can be updated and used quickly. I find that this simple project can help with applying some knowledge I gained from work, and it’s also quite fun.

The tools used to achieve this goal are

  1. Docker and Docker compose, and personal DockerHub registry.
  2. A personal server, I bought a second hand Intel NUC computer below for cheap, and installed Ubuntu server to it.

An Intel NUC, cheap and does not take much electricity.

3. Cloudflare tunnels: https://www.cloudflare.com/en-gb/products/tunnel/

4. Some cheap domain name bought from Namecheap

The application is containerized into two simple containers, just one DotNet WebApp container and one MySQL container, the database is stored in a separate volume so data is not gone when container is down.

In order to access the application from the Internet, I use a cheap domain name I bought for personal use. Then install Cloudflare tunnel daemon to the Intel NUC, then log in to Cloudflare and configure traffic to the domain to forward to the Intel NUC on HTTP port 8001, Cloudflare automatically enable TLS for the domain.

Port 8001 of the server is mapped to port 80 of the WebApp container, the WebApp container is configured to connect to the MySQL database instance in another container.

The cycle to build, update and deploy is manual but is fairly easy, all is accomplished using a docker-compose.yml file.

The process is like this

  • In development machine, run docker compose build to build local images.
  • Run docker compose push to push the images to personal DockerHub registry.
  • Use the same docker-compose.yml file in the NUC and just run
    • docker compose down
    • docker compose pull
    • docker compose up -d

And Docker will pull the latest images and start the containers, and they are available right away on Internet through Cloudflare tunnel.

It’s a good start for quick deployments/dogfooding and development.

Simple under-development home cooking planner.

Dev log [DotNetCookingPlan]: Revisiting meal planner

Recently I have been trying to study new technologies after changing to a new job and was quite tired. I tried to study “properly” by taking notes while reading books or watching course videos. It was not quite fun and not really the approach I used when I was a kid exploring the world of computer programming.

I decided to throw away my to do list, kanban board, etc and just jump into building something that I would actually use. I remember fondly of experimenting and using programming knowledge to build something either “cool” or useful, and it was very effective in learning new programming languages and frameworks.

I remember it was really boring attending lectures at University, and studying was not very effective. I guess the human brain needs rewards and feeling little successes to carry on doing or studying anything.

It’s the same story with learning to play an instrument, it’s very boring to follow some music sheet that the instructor gives you, but if you try to play some music that you actually like, it’s much easier to motivate yourself to continue practicing and learning.

I decided to rewrite the Meal planner application, this time as a full ASP.NET Core application using Razor pages. I took notes and studied ASP.NET Core before but the knowledge went away after long time.

The simple application will be improved incrementally and eventually be deployed inside a Kubernetes “cluster” hosted in a small personal server. Making this small application will allow me to

  • Have some practice in using Razor pages.
  • Revisit using Entity Framework Core with Code first migration, using MySQL database engine.
  • Containerize the application in a few containers using Docker and docker-compose, with volume mounts.
  • Repackage and deploy the application to a Kubernetes cluster, perhaps first by creating K8s manifests, and then by writing Helm charts.
  • Maybe eventually setup some CI/CD.

And the bonus is that I will use this project to plan my weekly cooking and grocery shopping, which is useful.

After this is done, I may improve on this project by converting it to ASP.NET Core web project with React.js. To revisit my React programming knowledge. However, to be more focused, I may not invest too much into frontend programming anymore.

Dev log: Finance and expense management application

After thinking hard and long about what kind of side project to do next and couldn’t think of a “good” and “unique” app idea, I decided to make my own expense management app. The goal is just to add to my portfolio of applications and learn new frameworks and tools as I develop it.

I have been using Expense Manager on Android for years, however the problem with using other people’s app is that there is a limit to how much I can customize it to suit my use the best. The way I use it, I only log my expenses every day and export it as an Excel file to review once a month.

It does not have an option to show the budgets I set every day/month right on its home page so that I can stop myself from overspending or being able to spend without feeling guilty whenever I log a new entry. In order to view my budgets, I have to tap some links. I would prefer it to remind me of how much I have spent and how much I can spend upfront when I open the app.

So I have been developing this application. The purpose is to learn all the following technologies in one go

  • Golang with Gin framework for backend, a RESTful Web API service.
  • MongoDB for data storage.
  • ReactNative for frontend.
  • Possibly scalable system using containers and kubernetes.

The application will have the following features

  • Offline first design with auto sync with server. AKA Google Keep. Conflicts will be resolved automatically.
  • Scalable storage using MongoDB.
  • Charts and summaries, budgets, etc. Simple and targeted to financial minimalists.
  • Cross platform, support Web, Android and possibly desktop clients.

Development progress is slow because I’m learning all these technologies as I work on it.

At the moment, the UI looks quite bad, I follow the philosophy “Functionalities first, polish later”. Some screenshots are below.

MealPlanner is released

Finally, after half a year of work, MealPlanner is finally released for both Web and Android.

The web version can be accessed here: https://bruceng.com/mealplanner

You can also install the Android version by going to the Google Play Store: https://play.google.com/store/apps/details?id=com.bruceng.mealplanner

If you do home cooking for the family or even only for yourself, you may find it useful to use this app to help with creating shopping list and maintaining food inventory.