Arun Mani J

The Palace of Illusions

Arun Mani J (j.arunmani@proton.me) — Sun, 18 Feb 2024 16:53:23 +0530

This book was gifted to me by my friend Levi from a book club for Christmas. Thanks Levi.

The last few months, I have been reading a lot of books related to Mahabharata. May be not so a lot but a lot at least. The same story is presented with little variations and adjustments in different books and narrations. Nobody knows which is the true one, if it ever exists.

So this book, it narrates you Mahabharata from Draupadi (Panchaali)’s point of view. If you are new to Mahabharata, then I suggest you to not start from this book. Read any other Mahabharata book from a third-party perspective and then read this book.

This book? It is awesome. I read Mahabharata from Jaya by Devdutt Patnaik, which tries to present a neutral narrative of what happened with a list of variations at the end of each chapter. If you read my review for Jaya, you will realize how I loved the book for its short, to-the-point descriptions. But by sacrificing the adjectives, the words lack any emotions.

However, The Palace of Illusions is completely opinionated. It speaks from Draupadi’s view. So there is a good amount of expressions and feelings preserved inside the book. It also presents a cinematic style of stories where the twist or surprise is at the last paragraph or sentence.

Overall, I loved the book for its presentation. If you are new to Mahabharata, then some plots may not be so obvious to you. That’s why, this book is great once you know Mahabharata.

Plato and a Platypus Walk Into a Bar: Understanding Philosophy Through Jokes

Arun Mani J (j.arunmani@proton.me) — Tue, 30 Jan 2024 21:38:14 +0530

For a long time, I thought philosophy means having abstract vague questions that are highly opinionated in nature and don’t have a solid answer. It includes questions like what is the meaning of life?, is the action fair? etc. But then I felt like - okay, but what is there for someone to study it so seriously?. In short, I thought philosophy is a topic of conversation people have from time to time in boring evenings where they themselves know that there is no proper answer. Sigh.

Okay, turned out to be, philosophy is kinda that but not that. I asked my friend to give me practical examples of where philosophy is used. I asked her to give a problem and explain how philosophy is used to solve it. She tried explaining me that you can’t understand philosophy just like that and yes, I didn’t.

I found out this book from a bookclub. The name sounded good and I felt like it will help me understanding philosophy. Well, after reading I can say that the book didn’t help me with my original objective in anyway. But, at least, it told me that philosophy is so vast and yea, spawns some long boring evenings and end up confusing people like me.

So the book presents some topic and a lot of jokes. I expected the book to explain the concept and then explain it with the jokes or at least explain the joke with the concept. But, the book takes a lot of things for granted. There is no real explanation actually. The book discusses a few paragraphs about the topic and then presents jokes.

The jokes are popular too, some small differences and twists. So it is highly probable that you know the jokes already.

In conclusion, if you already know or learning philosophy, then this book can be a nice companion in the journey. But if like me, you are a fresher, then this book may not be ideal, unless you wanted a book of jokes.

Twenty Two Goblins

Arun Mani J (j.arunmani@proton.me) — Thu, 11 Jan 2024 19:45:33 +0530

When I was young, a friend of mine told me a story of three skilled men and asked me who is the best among them. I forgot what answer I gave. But I started liking such stories where the conclusion is left to the reader.

Vikram-Vetala is a story about a king who is tasked by a sorcerer to bring a vampire to him. The vampire tells the king a few stories. At the end of each story, the vampire asks the king a question. If the king knows the answer but doesn’t reply, his head will burst into pieces. If he answers, the vampire will escape back to its origin. So for every story, the vampire flies back and the king goes back to fetch it. This continues for about twenty two stories (or twenty five as per Wikipedia).

After reading Mahabharata and a few other stories of Indian mythology, I thought of reading Vikram-Vetala tales too. I used Wikipedia as a reference to suggest me books. First I tried Richard Burton’s retelling but it was verbose and its language not easy for me.

Then I tried Twenty Two Goblins. The book suited me well because it was short and the language felt like literal translation. Like, the author even translated the names of people and cities to English.

Most of the stories had a similar plot. A love affair and almost all of them a love at the first site. The answers by king were also just and reflective, though a few controversial as per modern trends. Anyway, the stories flow through the classical Indian rules of justice, hierarchy and authority, which is good enough to keep you day dreaming.

Both of these books are in public domain, so you can read them right away 😇.

The Midnight Library

Arun Mani J (j.arunmani@proton.me) — Mon, 18 Sep 2023 21:34:19 +0530

Every moment in life, we have an infinite amount of choices though most of them are hidden from our sight. This book is a reminder of the same.

General narration to story, everything was nice. The book has short chapters which is what people like me love.

A non-spoiling gist of the story is it is about parallel world kinda life where each choice branches to a different one. The protagonist gets to live most of those different lives of her and realize herself.

I really like how this book has actual story instead of beating around the bush with boring text on depression etc. That’s a big plus.

Once you read the first few chapters, you will be able to predict the climax. But still reading the book fully is needed to understand the why of the ending.

In short, it is a very good story and a quick read too.

State of Stateless - DebConf 2023

Arun Mani J (j.arunmani@proton.me) — Sun, 17 Sep 2023 21:29:49 +0530

I presented this talk in DebConf 2023, Kochi, India.

Below is the gist.

Debian is one of the most used Linux operating systems. While RPM based OSes like Fedora, OpenSUSE are moving to Transaction-Rollback style package management, the scenario in Debian is still untouched.

In our talk, we explore and propose a user-perspective view on how a transaction based system can be introduced in Debian. We discuss about various technologies like layering of file system, taking advantage of BTRFS, and snapshot algorithms etc.

We also shed light on the new trends in Linux ecosystem like Flatpak, Nix, Guix and similar isolated package management systems.

Our talk is aimed at everyday users of Debian as the main motive of Debian is to ensure stability, having a deterministic and reproducible packages ensures that Debian systems can be deployed and used by users with least technical knowledge. This also means that Debian can be used in enterprise institutes with a stable base and fully-flexible changes to the packages in it. This can allow reduced-downtime updates of applications and services.

In short we enhance Debian’s 007 principle. 0 stability issues, 0 security issues and 7000 reasons to use Debian.

We highly believe that this talk will act as a stepping stone in bringing up the features that shall enhance the traditional operating system metaphor.

Creating Blurred Background Image using ImageMagick

Arun Mani J (j.arunmani@proton.me) — Sun, 13 Aug 2023 09:55:41 +0530

I get it, the title is confusing. Read along and you will understand.

If you are new to the concept of images. An image of size 100x200 means the image is 100 pixels wide and 200 pixels in height.

Aspect ratio is the ratio of its width to height. For an image of size 100x200, the aspect ratio is 1:2 or 0.5. It means, for every for every unit of width, the height must be twice the unit.

Introduction

We want to resize an image of arbitrary size such that

It matches aspect ratio x:y.
The aspect ratio of the original image is also respected.

Okay, say you have an image of size 320x428 and you want to resize it to 600x300.

Photo by Alena Koval: https://www.pexels.com/photo/orange-tabby-cat-on-brown-knitted-textile-982300/ See how the cat is concerned about its size!

You can do a brute force style of resizing the image. But.

Oh no, the cat lost his cool

As you can see, the forced resize gives off ugly results. And depending upon the nature of the image, it can even become disastrous. Like, you don’t want to do a forced resize of a map or plot!

The Need

Now you might ask — why do resize at all?! Because the world expects you to 🤷. For me, it was my website. If you go to the homepage, you will see how the article and library cards are arranged neatly.

It is because I have blurred their background and resized them to a fixed dimension of 640x360. If I didn’t do the resize,

The cards will be of different size each.
There will be an ugly empty space around images that don’t fit the card’s size.

As an example, see the below image. There are white spaces around my cat’s image.

For a website like mine, images come from various sources and they all have a different aspect ratio and size. So doing a forceful resize without preserving the original image’s aspect ratio is not good.

Just Crop?

Yes, you can simply chop of the segments and shape your image to the size you like. But the problem with cropping is that it works only when your image’s size is greater than the expected size.

You can enlarge the image but it leads to broken images. (Just try zooming a very small image, and you will see how the pixels start breaking with every zoom).

And obviously, cropping leads to the loss of data.

Blurred Background Image

Let me restate our objectives. We want to resize an image of arbitrary size such that

It matches aspect ratio x:y.
The aspect ratio of the original image is also respected.

The solution I use is like this

Forcefully resize the image to the desired size, call it A.
Blur A enough to cover the broken pixels.
Do a ratio-preserving resize of the original image, call it B.
Put B over A and call it C.
C is the resultant image.

You might have a question now — what is ratio-preserving resize? Say you have an image of size WxH and your target image must be of size PxQ. A ratio-preserving resize scales the original image such that its size is less that PxQ. That is, the resized image of size MxN will satisfy M <= P AND N <= Q AND M/N == W/H.

Taking back our cat example, the original image is of size 320x428. If we do the ratio-preserving resize to 600x300, then it becomes 224x300.

>>> 320/428
0.7476635514018691
>>> 224/300
0.7466666666666667

Both of their ratio is approximately equal to 0.74. Also see how our new image fits within the 600x300 requirement.

We can impose this image on the force-resized-and-blurred image. That’s it, we get a clean image of desired size.

Why Blur?

Instead of blurring, you might want to use a solid color like white, black or even transparency. The issue with the constant colors is that they are not adaptive. For example, my site has both light theme and dark theme. Depending upon your device settings, it chooses the color theme. Now, if I used a solid color of white, then users of dark theme will find it ugly. Similarly, users of light theme will find the black color background ugly.

Transparency is not even option because it takes us back to original problem of space around the image!

But blurring an image means the background is filled with the colors of the original image. If your image has a pink background, then the target image is also filled with pink background.

Blurred background images are not anything new. They have been there for a longtime. There are websites that lets you do it. There is a 110% chance that your favorite image editor already has a way to do it with less number of steps.

If the idea is still unclear, then have a look at this Graphic Design question - Is there an automatic way to resize images to a specific dimension. (blur background and not crop image).

Automating It Using ImageMagick

Actually, the main aim of the article is to tell you how to create the blurred background using ImageMagick.

If you don’t know what ImageMagick is, then quoting from their website;

ImageMagick® is a free, open-source software suite, used for editing and manipulating digital images. It can be used to create, edit, compose, or convert bitmap images, and supports a wide range of file formats, including JPEG, PNG, GIF, TIFF, and PDF.

But why ImageMagick? Because;

One of the key features of ImageMagick is its support for scripting and automation. This allows users to create complex image manipulation pipelines that can be run automatically, without the need for manual intervention.This can be especially useful for tasks that require the processing of large numbers of images, or for tasks that need to be performed on a regular basis.

If you go through the website, you will realize how ImageMagick is one of the best software that has been ever written.

The commands I used here are for ImageMagick 7. They should work fine with version 6 though. Depending upon your installation and version, the executable could be either convert or magick

The Command

magick input.jpg \( -clone 0 -blur 0x9 -resize 640x360! \) \( -clone 0 -resize 640x360 \) -delete 0 -gravity center -compose over -composite output.webp

That is a pretty long command, but once you know the basics, it is very easy to understand. On an overview, we can say ImageMagick takes an image input.jpg, does some magic and saves the result as output.webp. You can replace input.jpg and output.webp with your image names.

Let’s break down the command now!

Image Sequence and Parenthesis

The best explanation I have read is from ImageMagick documentation itself. Check it out, however I will try my best too.

ImageMagick works on a list of images. It means, to it, you give a list of images, ask it to operate on them and save the result. The images can be referenced by their position which starts from 0.

Now the parentheses ( ) is used to perform a set of operations and append the result to the end of images list. It is a very powerful feature, because it allows you to create a pipeline only for a specific set of images. In other words, it saves you from creating temporary images and then merge them together.

With this in mind, our command can be imagined to be like,

[input]--+-BLUR-+--[input, blurred]--+-RATIO_RESIZE-+--[input, blurred, resized]--...

...--+-DELETE_0-+--[blurred, resize]--+-COMPOSITE-+--[output]

Blurring

\( -clone 0 -blur 0x9 -resize 640x360! \)

This command blurs the input image (which is at index 0). But to be precise, it doesn’t blur the input image but blurs a clone of the input image. It clones the image on index 0, blurs it and the resizes it to 640x360. The exclamation symbol ! is to do a forced resize.

The 0x9 is radius (0) and sigma (9) of the blur. More details can be found in blur’s manual page.

Similarly, see resize’s manual page and clone’s manual page.

Ratio-Preserving Resize

\( -clone 0 -resize 640x360 \)

Similar to blurring, we create a copy of the input and resize it to 640x360. Note how there is no ! symbol at the end. It means we are doing a ratio-preserving resize.

Delete Input

-delete 0

delete removes the image from the images list. (See the pipeline overview above). But, remember, it doesn’t delete the file in anyway.

We are deleting the input, because we don’t want it to be a part of the merge. Our merge should happen between the blurred and resized image.

Gravity

-gravity center

The blurred and resized images are of different sizes. We should now merge them. But the resized image should be at the center of the blurred background. gravity does that by moving the images to center.

Compose

-compose over -composite

The merge process is called compose.

There are a lot of ways to add two images together. Like you can add their colors or subtract them etc. We want the resized image to placed over the blurred image. This is done using compose over.

Finally, we perform the composition using composite.

The resulting image is saved as output.webp.

Conclusion

Applying the command to our cat image, we get,

The cat is happy now!

Creating such images are not rocket science, but having a handy command that works offline and lets you do a rich set of operations is excellent.

Hope you enjoyed the reading!

Jaya

Arun Mani J (j.arunmani@proton.me) — Thu, 03 Aug 2023 19:58:17 +0530

Since my childhood, I always wanted to read Mahabharta. I tried once in my school days but didn’t have a nice time thanks to the number of characters and the family tree.

During my semester break, I thought I should try again but with a different book. The main reason is that I wanted to read the philosophy.

However, most of the books were really huge and had the old English. Then I found this book in Quora.

If you are beginner and like me — just wants the story without beating around the bush, then this is book for you. There are no paragraphs of adjectives or simply talking. The author comes to the point and has very crisp sentences. Each of the chapter is very small and it manages to tell you the essence. Apart from that, each chapter comes with a list of notes that talks about the variations, contemporary nature etc.

Such a wonderful book in itself. Highly recommended.

Okay, my thoughts on Mahabharta itself? It is very cunning and smart way of bending the rules, especially in the warfare. It is awesome. Though I feel like it is injustice sometimes, the other voice in me says - it is what it is, everything got perfectly balanced. So, once again, really awesome.

The Graveyard Book

Arun Mani J (j.arunmani@proton.me) — Thu, 03 Aug 2023 19:42:54 +0530

This book was suggested to me by friend Mangesh.

The title of the book itself was interesting and had an aroma of fantasy around it. And as someone who loves fantasy like anything, I gave it a try.

The book has a very simple synopsis. A family is murdered by a skilled murderer but a baby manages to escape. This baby finds care in a graveyard by the very ghosts. But the murderer is still looking for the baby. So the story is all about how the boy (the grownup baby) meets the murderer and learns some really twisty secrets about why the murderer is very keen on killing him.

With such a simple overview, you can get a sorta story completion by reading the first and last few chapters. The intermediate chapters feel like some random happenings in the boy’s life. BUT. Only at the climax you get to understand why all these intermediaries were there in the first place. Each and everyone of them has a small contribution at the last.

The book has a nice narration and so you won’t be bored if you ready a chapter a day.

What disappointed me is that the ending was a little vague. It just leaves you incomplete without telling you what’s next?, as if there is a sequel.

Apart from that, the book was good!

Publishing Book with Emacs and OrgMode

Arun Mani J (j.arunmani@proton.me) — Mon, 17 Jul 2023 19:29:28 +0530

I coauthored my first book — Problem Solving with Python Programming ¹. The book was written entirely in GNU Emacs using Org mode and exported as PDF using LaTeX. As a way of filling the empty blog and also to thank the open source communities behind the software, I decided to write this article.

Brief About The Book (Can be skipped)

The book is written for the courses GE3151 Problem Solving and Python Programming and GE3171 Problem Solving and Python Programming Laboratory of Anna University. The syllabus is more about problem solving than Python and being a course mandatory for all the engineering branches (so not just Computer Science and Engineering), it doesn’t touch the deeper realms of Python like Object Oriented Programming (OOP) etc.

If you did the course wholeheartedly, then you would be able to replace Python as your desk calculator and use it as a handy tool for housekeeping tasks like mass editing of files etc.

We sticked best to the syllabus because

We didn’t want to overwhelm the reader (a student or teacher) with full blast of Python.
The important excuse — the book with the stick to syllabus philosophy itself drew about 500 pages. Adding more to it will increase the pages which proportionally increases the cost of the book.

However for the chapters I wrote and code snippets I did, I tried my best to advocate the principles I have learnt so far in my journey of using Python (my favorite programming language).

It Started In My First Year

It started in the year 2020, when I finished my school and entered Anna University Regional Campus Coimbatore for Bachelors of Engineering (B.E.) in Computer Science and Engineering (CSE). I happen to be lucky to have my class advisor as the Python course instructor — Dr. Preethi J.

Having learnt Python since my school days and also using Debian GNU/Linux, I was one of the very few in my class who signed up for B.E. CSE because of interest in computer science than other reasons like job availability, easy to graduate etc. If I wanted to write my college story, then it would be as lengthy as a book, but that is not what the article is about. So let’s move forward.

Because that I was the Class Representative during the first year, I established a good name in the college. Once I had to meet my class advisor Dr. Preethi J for something and that’s when she asked — “I and your senior Akila are writing a book on Python, would you like to join us?”.

Akila; she was my senior and I had already met her during the online senior-junior meet-up (yes it was during the pandemic, so Google Meet). Being someone who doesn’t easily say No — I accepted the invite.

How To Write A Book?

We divided the syllabus. I was to write a unit (the syllabus contains 5 units) and do the laboratory manual. I was in home (actually I was in home only for the first and half a year because of the pandemic) and free of any classes because of semester holidays. It was the time to start writing. But how?

Yes really, how do you write a book? I mean what software should you use? I have used Libre Office, Scribus, LyX etc. since my childhood but I was confused which one would be perfect. I was in no position to use Microsoft Word because I can’t install Windows on my laptop and I can’t pay for the software either. The online version needs a Microsoft account; which means — No thanks, I will use something else.

I already dropped Scribus because I didn’t remember doing anything usefully in it beyond some samples. So the first one in the list is Libre Office. The biggest advantage of LO is that it is free and open source. The other authors can also install it on their Windows OS. Moreover, if they decide to use Microsoft Word, I can import the .docx file in Libre Office. So in terms of collaboration, it is a perfect tool. Next the graphical user interface means easy to learn (actually it is discover-ability).

But I really didn’t like to use Libre Office to write a book. It felt too fragile.

Meme downloaded from https://www.memedroid.com/memes/detail/3510839

The meme applies for Libre Office too in my opinion. Moreover, having to click around the UI for everything felt annoying when you needed the same configuration in another document. Because if you used a base template for all documents and decided to make a small change in the template, now you have to make the change in all the documents too.

But apart from all these, the most important showstopper for me was that I didn’t manage to get syntax highlighting to work in Libre Office. I tried a bunch of add-ons and solutions of Stack Overflow but they didn’t work. What is a programming book without syntax highlighting? 🤷‍

Here Comes Emacs

So now the option is to consider LyX. But. This is where a few brain cells left in me woke up and said — Hey! If you are going to use LyX, why not use Emacs itself?. The beauty of LyX is that it gives you the beautiful LaTeX typeset PDF without having to learn LaTeX. But if that was it, Org mode can do it already. Instead of having to install a Qt application in my GNOME/Sway desktop, I can simply use my existing Emacs installation. I was already a diehard Emacs follower thanks to Doom Emacs and the open source community in general. So using Emacs should be easy for me.

Honestly speaking, looking back at the moment, the reason I chose Emacs that time is because of laziness to install LyX and the voice inside me that kept saying — Dude simply start writing, you can shift to other tools when needed. But now if you ask me if the choice was right — I can stand up and shout a BIG YES!.

Yes, using Emacs came with a massive set of advantages that was not visible at the time of choosing. I have summarized it in the last sections.

So Org mode and LaTeX

Meme downloaded from https://www.reddit.com/r/emacs/comments/fzwk6u/learning_curve_on_emacs/

Jokes aside, I’m not a power user of Emacs in any way. I have been able to use Emacs because of Doom Emacs, which takes care of providing an amazing workflow. I don’t write Elisp and my configuration (on-top-of Doom Emacs) is hardly ten lines I copied from various blogs and solutions to provide CUA Mode, enable paragraph filling etc.

The next big thing is Org mode. I have used it just because I have seen people making miracles with it. I tried maintaining a TODO or Agenda via Org mode but it failed. However I do use Orgzly in my phone to remind me of birthdays, but that’s a different story. I have exported Org files to PDF using pdflatex (available in Emacs via C-c C-e l p). The beauty of LaTeX with the best text-editing features provided by Emacs, makes Org mode a killer software.

I accept, the experience of using Org mode was not butter-smooth. I occasionally messed up indentation (Org mode uses indentation!) and had to switch to text-mode or open up the file in another text editor to fix the issue. Learning Org mode syntax and keybindings took its own time. Once I look up and use it, I felt confident but after an hour or day passes, I forgot the keybindings or syntax and had to look up again. In short, the muscle memory took a long time to build up.

LaTeX? Don’t ask 🙂. I have been able to use LaTeX only because of software like LyX and Org mode. I can proudly say that I know no LaTeX command by-heart before writing the book and also after writing the book. To me LaTeX belongs to the smartest of smartest people (no offense please!) and I don’t think I will learn it any day soon.

Org mode was good enough to hide 99% of LaTeX away from me and the remaining 1% was available by web searches.

Gathering my guts, I wrote my first unit (a single .org file). This is the template I used.

#+TITLE: Unit Name
#+OPTIONS: author:nil date:nil title:nil H:4 num:2

* Heading
… content goes here …

It produced a good looking PDF without the author and date (which was not irrelevant for a book). I had to tweak the limit of headings via H:4 num:2 because some low-level headings were converted to definition lists. Obviously this was with time-consuming web searches.

If you used the above template, you did get a page that looks like this.

But if you follow carefully, you must have noted that one of the reasons to leave Libre Office — syntax highlighting, is absent! So we have to enable syntax highlighting in the PDFs exported in Org mode. If you do a web search for Org mode export syntax highlighting, you see helpful Emacs Stack Exchange threads.

So you have to install Pygments and add the following to your Emacs config.

(require 'org)
(require 'ox-latex)
(add-to-list 'org-latex-packages-alist '("" "minted"))
(setq org-latex-listings 'minted)

(setq org-latex-pdf-process
 '("pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f"
 "pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f"
 "pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f"))

This gives us a syntax highlighted PDF like this.

Shaping for Lab Manual

With the above template, everything worked well for unit. But once I started writing lab manual, it became a little difficult because of two issues:

Maintenance of programs.
Creation of flowcharts.

Maintenance of Programs

Though Emacs provided syntax highlighting and automatic indentation while editing code blocks in an Org file, it is very much limited. Once your program goes past hundred lines, you wish there was full-fledged Python environment that provided auto-completion and formatting.

To aid in this task, Org mode has #+INCLUDE: file.py src python directive that adds file.py to the Org document as a code block. It means I can edit the file as an ordinary Python file and while exporting the PDF it will be properly included with all the goodies.

Creation of Flowcharts

Now comes the task of creating flowcharts. For someone who sucks at any kind of artistic creativity, making flowcharts using a GUI tool is very difficult because of the click-click process. I thought it would be great if there was a software that drew flowcharts from a text file containing the rules.

AT&T Labs Research faced the same issue and designed the golden Graphviz. After skimming over a few tutorials and understanding the syntax, I was able to make flowcharts.

Like any other Emacsen, I wanted a way to edit the .gv files inside Emacs and found graphviz-dot-mode.

One issue, I haven’t been able to resolve in Graphviz yet is making the edges come out of the vertices instead of anywhere in the shape. However I ignored it for all flowcharts except a few. For the few, I used draw.io (and draw.io is great!).

Getting a Publisher

The first year became second year and the second year became third year. We all became busy in our own works and the book writing came to a temporary halt because the syllabus was about to change.

Once the new syllabus came (which luckily had less changes) we thought to hurry up the process. To hurry up means to get a publisher, because once we knew someone is waiting, we would try to finish the work as soon as possible. We tried a few big academic publishers but they were reluctant to accept the proposal because the coauthors were students. Yes. They didn’t accept because they don’t take books from students except PhD students. However they were ready if we agreed that only our professor is the author. Means the book will have only our professor’s name.

Thankfully, our professor didn’t go with it. We then got help from college librarian and got contact of a publisher named Yes Dee.

The company’s managing director and technical editor met us in college and they were really happy to see that the book was written by students. So that was it, they accepted our proposal and we signed the contracts.

After many topics, our discussion came to the manuscript format. Our professor pointed to me and said that I’m using LaTeX to type the book. I took my laptop and showed them the units and lab manual I have written so far.

The publishing team said that since I knew LaTeX already, they would give me the guidelines for the book like fonts, layout, dimensions and I was to give them the final PDF as per the guidelines.

Yes 🙂. They assumed I know LaTeX. I till, the moment of the very sentence, thought that we can submit the manuscript is any legible format and publisher would take care of shaping it to the right layout.

May be my assumption was wrong or it was my luck, I do not know. I didn’t deny the offer because of two reasons:

If I denied because I don’t know LaTeX then my professor would be upset because I had convinced her LaTeX is the best option over Microsoft Word. I promised her that I would take care of any issues arising due to this choice.
I felt that if I did the formatting then I can make any changes required to their guidelines wherever deemed fit.

Whatever it was, I accepted and we were to submit the final PDF in a month. I still told them that I don’t know LaTeX much and they said they will help me. 😁

Sudden Fear

Since I was supposed to format and submit the PDF, I was stuck by fear and doubt if I will be able to do it. I didn’t use LaTeX directly but LaTeX on top of Org mode. So every issue I face has to be first checked against Org mode and then LaTeX.

This was the moment, on a fine Sunday, I was questioning myself if by using Org mode I have made the biggest mistake. Had I used LaTeX directly, then I can ask in LaTeX forums directly but with having Org mode as an interface I had to explain them Org mode too, which might affect the answers.

So I thought — okay, let us use LyX. Org mode already produced .tex files during PDF export, I thought of importing them in LyX and drop the dependency of having to use Emacs completely. I also convinced myself that it would allow other coauthors to collaborate as it is easy for them to use LyX than learn and use Emacs.

Importing the .tex file didn’t go smooth though. Some LaTeX commands inserted by Org mode didn’t compile in LyX. And I was already counting my luck. Okay! A new idea comes. Let us copy paste the paragraphs into a new LyX file itself and make the necessary formatting. It is a tedious job but I forced myself to do it. I tried a chapter but couldn’t go through it. Clicking the buttons, enabling syntax highlighting and so and so, it was a pain.

I gathered hope again. No, stop it. I told myself that whatever it is, I will stick to Emacs and finish it. If I couldn’t do it, I will let the publisher know. I don’t know what gave me that hope but it worked! Importing the entire book into LyX itself would have taken a week (remember, I have classes from 9 AM to 5 PM and other works too) and could have caused more harm than good.

Back to Emacs.

Merging Chapters

A lot changed after we met the publisher. Our professor said that breaking the five units into individual chapters would be a good idea. I agree because they are more easy to follow than grasping an entire unit. As the other coauthor was busy with her placements and I was assigned two more units. My professor said she will take care of solved questions and exercises for every chapter.

Writing has never been an issue for me, if needed I could have written the entire book because Python is my favorite language and I loved writing about it.

But, merging other people’s work was very hectic. It was .docx files from the other coauthors. I opened them in Libre Office, copy pasted them in .org file and reformatted them. This I did paragraph by paragraph. Copy in Libre Office. Paste in Emacs. Reformat the paragraph and replicate the bold, italics etc. Repeat.

Sometimes the .docx files had images and I had to save them using Libre Office Right Click -> Save and then insert it in Emacs. Images come with their own headaches. I had to resize the image and make sure its placement doesn’t affect the other text.

A little by little, I converted all the Word files to Org. After making the changes, I emailed the chapters to the editor at the publishing company. The editor, she must have had a lot of patience, reading through the text and marking up the changes. I on the other hand, fixed them. The grammatical corrections were easy. The changes regarding layout were trouble. It means I had to step into LaTeX world and hunt for the commands. It was a nightmare. This process went for around three times I think.

Final Steps

I created all the pages from the front page, copyright, acknowledgments to chapters in Org mode and exported them as PDFs. I then combined the PDFs together using Ghostscript.

After ensuring that all the corrections have been made, the book went for printing. After a month, we received the authors’ copies last week. It felt so great to have your time and efforts as a physical item in your hands.

I smiled at the book and told myself — Yes! I did it. I did it using Emacs and Org mode. It is not a flex but a reminder that I did that even though I doubted myself. Looking back, it feels like a lovely nightmare. 🙈

The Right Choice

Did I make the right choice by using Emacs and Org mode instead of something else? With the book in my hand, I can say yes. Doing it this way gave its own set of advantages.

Everything is text. Org files are text files, they can be edited in any text editor, though Emacs has the best interface to interact with Org files. Text is the universal format. Because of this fact, I was able to edit the Org files in various ways. Like, I could do some complicated search and replace using Python etc.

The best of all is that any change in template is just addition or removal of few lines of text. So I can simply copy paste it in all the files. Add this to the powerful Emacs commands, you have everything in your reach.

Now compare it with using word processors like Libre Office. I think they save their files in some form of XML. Every change would be accompanied with a number of mouse clicks, pop-ups, menus etc. Going from the simple text format to a word processor would have been like tying my hands.

But every advantage comes with its own disadvantage. Using Emacs made it a sort of vendor lock-in. Only I could do the formatting work because the other coauthors used Windows. Installing Emacs, LaTeX in Windows and making them learn it might be difficult.

During the corrections phase, my senior and coauthor Akila helped with grammatical corrections. I asked her to install VSCode Org mode plugin to at least get the syntax highlighting of Org files. But she was limited to grammatical corrections. Anything beyond like layout or formatting needed LaTeX knowledge, so I had to it. This way I ended up being the pressure bearer.

May be if there was a web app of Emacs with all the goodies, it could have helped but none existed when I searched.

The next disadvantage is that most of the commands, Elisp code I used where blind copy-paste with a very little knowledge of what’s happening underneath. If something broke, I had to search the error and fix it. Because I didn’t know LaTeX or Org mode well, it was a slow process.

Had there been a major version upgrade in LaTeX or Org mode such that the commands from five to ten years old didn’t work, then I would have been in a BIG TROUBLE.

But luckily nothing of that sort happened. Good software lasts long. Emacs and LaTeX did for me. I’m happy for them.

Technical Details

If you wanted me to shut up and show the code, then there you go. I will try my best to explain the commands and organization of the source files.

Setting up Emacs

I used Doom Emacs. Though I haven’t tried, nothing I did is exclusive to Doom Emacs and should work with other variants too as long as Org mode exists.

Setting up LaTeX

The next step is to install LaTeX. The way of installation depends upon your OS. I started from a standard install and installed extra packages as needed. If you are using Debian GNU/Linux, then the following command should suffice.

sudo apt install texlive texlive-latex-extra texlive-plain-generic --no-install-recommends

I have added --no-install-recommends to avoid cluttering with unnecessary packages like X11 tools.

Miscellaneous Packages

Once you have installed LaTeX, you need to install some other helpful packages.

sudo apt install ghostscript imagemagick mupdf-tools python3-pygments

Ghostscript for merging PDFs.
ImageMagick for editing images.
MuPDF for extracting table of contents.
Pygments for source code highlighting.

Enabling Syntax Highlighting

To enable syntax highlighted code blocks in PDFs, you need to add the following lines to your Emacs config.

(require 'org)
(require 'ox-latex)
(add-to-list 'org-latex-packages-alist '("" "minted"))
(setq org-latex-listings 'minted)

(setq org-latex-pdf-process
 '("pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f"
 "pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f"
 "pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f"))

Organization

I wrote each chapter in a separate Org file. The advantage is easy to maintain but the disadvantage is that I had to manually set the page number of the first page and then merge them properly.

The Template

The .org file I used as template for all the chapters is as follows.

#+title: 4 Control Flow
#+LATEX_CLASS: extarticle
#+LATEX_HEADER: \usepackage[paperwidth=152.4mm,paperheight=228.6mm]{geometry}
#+LATEX_HEADER: \geometry{verbose,tmargin=25.95mm,bmargin=19.3mm,lmargin=16.2mm,rmargin=16.2mm}
#+LATEX_HEADER: \setcounter{page}{143}

#+LATEX_HEADER: \usepackage{minted}
#+LATEX_HEADER: \usemintedstyle{bw}
#+LATEX_HEADER: \setminted{frame=lines,breaklines=true}

#+LATEX_HEADER: \usepackage{fancyhdr}
#+LATEX_HEADER: \pagestyle{fancy}
#+LATEX_HEADER: \fancyhf{}
#+LATEX_HEADER: \fancyhead[LE]{Problem Solving and Python Programming}
#+LATEX_HEADER: \fancyhead[RE]{\thepage}
#+LATEX_HEADER: \fancyhead[LO]{\ifnum \thepage=143 \textbf{\Huge{4}} \else \thepage \fi}
#+LATEX_HEADER: \fancyhead[RO]{\ifnum \thepage=143 \textbf{\Huge{Control Flow}} \else Control Flow \fi}

#+LATEX_HEADER: \renewcommand*{\thesection}{4.\arabic{section}}
#+OPTIONS: author:nil date:nil toc:nil title:nil H:4 num:2

#+LATEX_HEADER: \tolerance=1
#+LATEX_HEADER: \emergencystretch=\maxdimen
#+LATEX_HEADER: \hyphenpenalty=10000
#+LATEX_HEADER: \hbadness=10000

* Verbatim
#+begin_src text
Monospaced Text
#+end_src

* Code
#+begin_src python
for i in range(10):
 print("Hello World")
#+end_src

* Unnumbered Heading
:PROPERTIES:
 :UNNUMBERED: t
:END:

Let us go through it part by part.

#+title: 4 Control Flow
#+LATEX_CLASS: extarticle
#+LATEX_HEADER: \usepackage[paperwidth=152.4mm,paperheight=228.6mm]{geometry}
#+LATEX_HEADER: \geometry{verbose,tmargin=25.95mm,bmargin=19.3mm,lmargin=16.2mm,rmargin=16.2mm}
#+LATEX_HEADER: \setcounter{page}{143}

The first line sets the chapter title. Next to it we set the LaTeX class to extarticle which allows extra font sizes. See this post for more details.

Then we set the page dimensions followed by the margin at each side. The top margin is more than bottom margin because top of the page contains the chapter header. (Reduce it and see, to know what I’m saying). Finally we set the page number of the first page in the chapter.

Determining the page number is a two-way process. First you compile all the chapters with page counter as 1. This way you get to know how many pages each chapter has and note it down in a file. After that, you set the page counters to the values noted.

#+LATEX_HEADER: \usepackage{minted}
#+LATEX_HEADER: \usemintedstyle{bw}
#+LATEX_HEADER: \setminted{frame=lines,breaklines=true}

Here we tell LaTeX to use minted for code snippets. This will take care of printing the code with syntax highlighting. I’m using black and white bw style because the book would be printed in black and white. Then we put each code block between two lines. breaklines=true makes sure LaTeX breaks the lengthy lines into newlines (that is wrapping) instead of letting them run past the margin.

#+LATEX_HEADER: \usepackage{fancyhdr}
#+LATEX_HEADER: \pagestyle{fancy}
#+LATEX_HEADER: \fancyhf{}
#+LATEX_HEADER: \fancyhead[LE]{Problem Solving and Python Programming}
#+LATEX_HEADER: \fancyhead[RE]{\thepage}
#+LATEX_HEADER: \fancyhead[LO]{\ifnum \thepage=143 \textbf{\Huge{4}} \else \thepage \fi}
#+LATEX_HEADER: \fancyhead[RO]{\ifnum \thepage=143 \textbf{\Huge{Control Flow}} \else Control Flow \fi}

It is time to style the headers of each page using fancyhdr. The layout of the header is simple. The publisher told me that every new chapter should start at the right hand side. If the chapter ends in right, then I was supposed to add a blank page, so that the next chapter starts in the right.

In the configuration above, L means left, R means right, E means even and O means odd. The header is divided into three parts — left, center and right. This header can be in an even page or odd page.

If you use right-odd numbering system (where the right hand side contains odd page numbers), then every new chapter will be on an odd page. So the layout I used is as follows:

On every even page (left hand side), put the book title at the left corner and the page number at the right corner.
For every odd page,
- If the page is the starting page of a chapter \ifnum \thepage=143, then we put the chapter number at the left corner and chapter name at the right corner in a huge font size.
- Else, the page number at the left corner and chapter name at the right corner.

Here 143 is the page number at which the chapter starts and has to be set properly as we did with the page counter.

#+LATEX_HEADER: \renewcommand*{\thesection}{4.\arabic{section}}
#+OPTIONS: author:nil date:nil toc:nil title:nil H:4 num:2

The first command adds 4. to every section number. Here the number 4 denotes the chapter number. So sections in this chapter come out like 4.1, 4.1.1, 4.2 etc.

The next command turns off displaying of author, date, table of contents and title. Then it sets the depth of sections.

#+LATEX_HEADER: \tolerance=1
#+LATEX_HEADER: \emergencystretch=\maxdimen
#+LATEX_HEADER: \hyphenpenalty=10000
#+LATEX_HEADER: \hbadness=10000

All these four commands work toward one common goal — to prevent hyphenation of words altogether. This was a requirement from the publisher. These commands made sure that if a word had to be hyphenated, then it must be in the next line without any breakage.

* Verbatim
#+begin_src text
Monospaced Text
#+end_src

* Code
#+begin_src python
for i in range(10):
 print("Hello World")
#+end_src

* Unnumbered Heading
:PROPERTIES:
 :UNNUMBERED: t
:END:

These are not part of the template but to help me remember the syntax for frequently used content. The important one is :UNNUMBERED: t which made sure that the corresponding heading isn’t numbered in the output. It was necessary for the headings like solved questions and exercises at the end of the chapter, which shouldn’t be numbered.

Creating Table of Contents

The issue with table of contents and other front matter is that they use Roman numbering. Because of this, \ifnum \thepage=143 command to set the header depending upon the value of page number (\thepage) doesn’t work with Roman numbers.

I tried searching the web but didn’t find any solution that works with fancyhdr. Since the number of pages is less in number, I split the ToC into two Org files. The first Org file has a header and the other Org file do not.

For the listings of the chapter itself, I merged all the chapters using Ghostscript and then extracted the table of contents using MuPDF. Then I used a Python script to convert the ToC to the section listings.

#+title: Table of Contents
#+LATEX_CLASS: extarticle
#+LATEX_HEADER: \usepackage[paperwidth=152.4mm,paperheight=228.6mm]{geometry}
#+LATEX_HEADER: \geometry{verbose,tmargin=25.95mm,bmargin=19.3mm,lmargin=16.2mm,rmargin=16.2mm}
#+LATEX_HEADER: \pagenumbering{roman}
#+LATEX_HEADER: \setcounter{page}{7}

#+LATEX_HEADER: \usepackage{fancyhdr}
#+LATEX_HEADER: \pagestyle{fancy}
#+LATEX_HEADER: \fancyhf{}
#+LATEX_HEADER: \renewcommand{\headrulewidth}{0pt}

#+OPTIONS: author:nil toc:nil title:nil date:nil H:4 num:2

#+LATEX_HEADER: \tolerance=1
#+LATEX_HEADER: \emergencystretch=\maxdimen
#+LATEX_HEADER: \hyphenpenalty=10000
#+LATEX_HEADER: \hbadness=10000

#+BEGIN_EXPORT latex
\begin{center}
\textbf{\Large{Contents}}
\end{center}
#+END_EXPORT

#+BEGIN_EXPORT latex
\contentsline {section}{\numberline {}{\emph{Preface}}}{xix}{section}%
\contentsline {section}{\numberline {}{\emph{Acknowledgements}}}{xxi}{section}%

\begin{center}
\textbf{Problem Solving with Python Programming}
\end{center}

\contentsline {section}{\numberline {1}{Computational Thinking}}{3--94}{section}%
\contentsline {subsection}{\numberline {1.1}Fundamentals of Computing}{3}{subsection}%
\contentsline {subsubsection}{\numberline {1.1.1}Variables}{3}{subsubsection}%
\contentsline {subsection}{\numberline {1.2}Identification of Computational Problems}{3}{subsection}%
\contentsline {subsubsection}{\numberline {1.2.1}Decision Problems}{3}{subsubsection}%
…
#+END_EXPORT

Everything is same as the normal template except the fact that I wrote LaTeX directly for the listings. It is pretty hard-coded, so every time the pages in the chapters change, I have to change the table of contents too.

#+BEGIN_EXPORT latex
\begin{center}
\textbf{\Large{Contents}}
\end{center}
#+END_EXPORT

This code block is present only at the title page but for the remaining I use the below template.

#+title: Table of Contents
#+LATEX_CLASS: extarticle
#+LATEX_HEADER: \usepackage[paperwidth=152.4mm,paperheight=228.6mm]{geometry}
#+LATEX_HEADER: \geometry{verbose,tmargin=25.95mm,bmargin=19.3mm,lmargin=16.2mm,rmargin=16.2mm}
#+LATEX_HEADER: \pagenumbering{roman}
#+LATEX_HEADER: \setcounter{page}{8}

#+LATEX_HEADER: \usepackage{fancyhdr}
#+LATEX_HEADER: \pagestyle{fancy}
#+LATEX_HEADER: \fancyhf{}
#+LATEX_HEADER: \renewcommand{\headrulewidth}{0pt}
#+LATEX_HEADER: \fancyhead[LE]{Contents}
#+LATEX_HEADER: \fancyhead[RE,LO]{\thepage}
#+LATEX_HEADER: \fancyhead[RO]{Contents}

#+OPTIONS: author:nil toc:nil title:nil date:nil H:4 num:2

#+LATEX_HEADER: \tolerance=1
#+LATEX_HEADER: \emergencystretch=\maxdimen
#+LATEX_HEADER: \hyphenpenalty=10000
#+LATEX_HEADER: \hbadness=10000

#+BEGIN_EXPORT latex
\contentsline {subsubsection}{\numberline {1.9.2}Shapes}{34}{subsubsection}%
\contentsline {subsection}{\numberline {}Solved Problems}{35}{subsection}%
…
#+END_EXPORT

Designing Front Matter

The other pages present in front matter includes title page, copyright, table of contents, preface, acknowledgments etc. For those all, I used the following template.

#+title: Dedication
#+LATEX_CLASS: extarticle
#+LATEX_HEADER: \usepackage[paperwidth=152.4mm,paperheight=228.6mm]{geometry}
#+LATEX_HEADER: \geometry{verbose,tmargin=19.3mm,bmargin=19.3mm,lmargin=16.2mm,rmargin=16.2mm}
#+LATEX_HEADER: \thispagestyle{empty}
#+LATEX_HEADER: \pagenumbering{roman}
#+LATEX_HEADER: \setcounter{page}{5}

#+OPTIONS: title:nil toc:nil author:nil date:nil H:3 num:nil

#+BEGIN_EXPORT latex
…content…
#+END_EXPORT

Finishing Up

With the above templates I created the front matter and content PDFs. Then combined them using Ghostscript. For about 500 pages, it came about 5 MB.

Conclusion

That was it. This was my path of ambiguities and errors to successfully publish the book. Success of the book itself is not known yet but I was more excited about the launch phase itself.

Will I use Emacs again if I had to publish another book? Pretty sure, once you are comfortable with the process, you are ready for another dose 😉.

I also frankly admit that there could have been a better way but I did what I could with the tight deadline. I’m open to ideas on how it can be improved 😁.

For a formal conclusion, hear me out. Book writing (means the text itself, not the formatting, LaTeX or Org mode) is a process where you (re)evaluate what you understood. I have heard the saying that if you can’t explain something to someone then you haven’t understood it in the first place. Leaving aside the rigorous rules, the saying is really apt.

Explaining the concepts like string slicing with negative indices were not so easy. I rewrote the paragraphs again and again until I felt confident that the words were easy. Sometimes I had to cross check my understanding to ensure that I don’t provide wrong information.

Definitions are a thing on another level. In my opinion a definition or a group of definitions must be as independent as possible. That is, you don’t want definitions to link to definitions and have the reader turn pages to connect them.

At such situations, I have found that using two definitions work better. First definition that is isolated and independent of other definitions and a second definition that links to other definitions.

Thanks for reading! If you have any comments, questions or suggestions feel free to reach me out using the links in footer. Have a nice day.

Footnotes

The website shows only the professor name because the publisher said that their convention is to use only the first author for copyright and website display. ↩︎

The Nightingale

Arun Mani J (j.arunmani@proton.me) — Sat, 15 Jul 2023 14:26:26 +0530

One of the worst things in this world is being punished for your birth and background to which you had no control over. This includes all sort of discrimination based on your caste, color, race, name, family etc.

The next worst thing is suffering because of someone else’s greed and being pulled into trouble because they think it is right. This includes all the wars that have been taken millions of lives because of someone’s ego, pride and wrong-thinking.

I have already read Man’s Search for Meaning and it gave me an account of the horrors in the concentration camps.

This book was different. Assume we can divide the people into three groups during war - those suffering by fighting for their country, those suffering by being in prison and finally those left in their homes to suffer with the hell caused by war. The Nightingale belongs to the last category. It tells the story of two sisters about their lives and acts during the World War II.

Though it is fiction, the book had a sense of realism to it. It is based on a true war bought by true people which took true lives of innocents. The narration was beautiful. I at sometimes, lost myself in thoughts on what I would do if a war broke out. Or sometimes, I question what my action would be if I was in the place of the protagonists. A war changes everything.

I loved reading the book. I had a goal of reading five chapters a day but soon began to cross the limit and read as much as I can. I feel like the last few chapters were rushed up and some more details could have been added to ensure the smooth flow. Apart from that, this was a joyful reading of a past nightmare.

Looking for Alaska

Arun Mani J (j.arunmani@proton.me) — Thu, 29 Jun 2023 15:02:36 +0530

I forgot this book completely. Yesterday at 2 AM, while thinking of completing the Malgudi Days, I suddenly got reminded of it.

So only ~200 pages? I can finish it.

Reminder to self - be careful when reading a love story 🥹.

This is really a paradoxical love mystery, and it reminded me of Norwegian Wood. The narration was simple and a little more explicit with American slang. Yea fine, Malgudi Days had Indian slang and so why not 😉.

The first part of the book was real fun. Full of events and happenings, it had the visual feel attached to it.

The second half became a real slow wheel. Grinding the same thing again and again and you feel like all the grinding was for nothing by the time you read the last page. If this was an Indian film, then the second part would be two songs and a 5 minutes scene.

But at least, the aroma of the first half, forces you to read the second. And the first part is good enough to tickle you with memories or worry you with the lack-of (like me) depending upon what kind of childhood you had and the school 😎.

Malgudi Days

Arun Mani J (j.arunmani@proton.me) — Thu, 29 Jun 2023 14:55:03 +0530

I had my exams this month, so I didn’t read much, though I tried my best to read one story a day.

Once you ignore the foreword by some author (gosh it covered pages and did more harm to me), this book is smooth.

The stories don’t have any path, you just read and it finishes. So what kept me reading? It is the narration.

It gives you the feel of a village folklore, of the daily happenings in your neighborhood. Yes the story happened in the days where there was no phone or so and yes I don’t belong to that era. But born in a relatively remote area with a parents of rich childhood memories, who here and there narrated me their stories, Malgudi Days gave me the equivalent feel.

Reading through the stories without any objective felts like a usual gossiping. You stand in your home and your neighbor comes and tell you of something in the town. This book gave me that experience.

A few of the stories were interesting. A few simply left a sad impression, for example the snake charmers, knife sharpeners, cobblers etc. The story described the real life. No hero, no rich mansion heroine, no action-romantic sequence. It was like a hard truth. Take it or forget it.

If you are someone from the Indian locality (not the urban region), then you might get a strange nostalgia reading this book. At least, its stories are 10 pages roughly. So reading one a day won’t be a big deal either.

The Shadow of the Wind

Arun Mani J (j.arunmani@proton.me) — Tue, 25 Apr 2023 23:52:20 +0530

After a long time (may be not so long), I read a book with 400+ pages. I should say that the efforts paid were completely worth. I had to read the book amidst a busy schedule like project submissions and internal exams. So I spent an hour or so whenever I felt like I need to forget the real world. Sometimes I even forced myself to read the book because the story really pulled me in. 🙈

What’s the speciality of the book? The protagonist (actually, the book doesn’t have a single one) is a hero with no magical powers. He goes through a lot of hard-times and disappointments. Almost every character you encounter in the story has a very pitiful past in one way or the other. May be, this environment gave the book a realistic feel.

As a personal favorite, all chapters were with ten pages, which is satisfying if you are someone who counts progress based on number of chapters you have read. 🙂

I don’t want to spoil the story, but if you are a fan of thrillers, twists and especially love, then this book can be a great time sucker. Bear in mind, the love you are about to see also includes some heart breaks.

Finally, such a well written book and I enjoyed it. 😊

Norse Mythology

Arun Mani J (j.arunmani@proton.me) — Fri, 24 Mar 2023 22:42:10 +0530

Honest disclosure, I’m a huge fan of fantasy and mythology, so I could be over positive about this book.

After having a total failure on The Starless Sea, this book served as a big aid in getting me back to reading™. I love mythology so crazily, and no matter how much expressive they are, I always feel like that’s not enough. But that’s just another story.

Now about this book- it was excellent. It has a little number of chapters where each chapter is a short story involving Gods and giants. The stories were really short, nothing verbose or boring. The number of pages in the book itself is less, so a professional book dragon 🐉 can eat up from cover to cover within a day.

It starts, like any other mythology story, with the narration of world creation. After that a few stories about the famous Gods like Odin, Thor, Loki etc. The middle stories are all about other Gods and other happenings. The final chapters describes the end, which I guess, if I explain, would become a spoiler.

What I liked more about the stories (or the Norse mythology itself) is that most of the stories have a connection with the life we lead now. It could be about cheating or wordplay or underestimation etc.

Finally, if you are in need of short stories of short duration to fill your break time, then I highly recommend this book. You need not remember the previous stories to read a new one as the stories are not much connected. So it reduces the burden on your memory especially if you are a person like me who forgets everything easily. 🤓

A great book, enjoyed every word of it.

Let's Make A Silly JSON-like Parser

Arun Mani J (j.arunmani@proton.me) — Wed, 01 Mar 2023 00:00:00 +0000

Hey there everybody! To give a tl;dr;, we are going to make a silly parser for JSON-like syntax. I call it silly because the main aim is to simply write one for pure fun and not focus on any blazing efficiency. Second of all, it is JSON-like because, the syntax closely resembles JSON but it isn’t exactly one. But we drop the suffix like for the sake of brevity.

Understanding Our Data

Before writing our parser, we must know what our data is and how it looks like. If you know JSON, then there won’t be much discrepancies.

Our notation supports numbers, strings, lists, JSONs and null.
The numbers can be positive or negative. Both integers and floats are supported but only in base 10 representation. Examples include 123, 2.17, -3.14, +20.2.
Strings must start and end with double quotes, like "Alice", "Hello, World".
A list consists of comma separated elements within square brackets [ ]. The elements can be heterogeneous, so [1, 2.3, "hello", [0, 10], null] is valid.
JSON is enclosed inside curly braces { }. It contains key-value pairs of the syntax key: value. Colon : is used to separate a key from value. Keys must be strings. Value can be any valid type. The key-value pairs are separated by comma. Examples include {"name": "Joe", "age": 25, "numbers": [10, 20, 30]}.
null is included as it is. Fully lowercase and no quotes.
Space can be used deliberately. For numbers, there can be multiple space between the sign and value, for example + 19.27 (we will see why later.) Here space means any whitespace character including newline, tab etc.

Phew! With the above seven rules, an example data could be as follows.

{
 "key": "value",
 "number": -3.14,
 "list": [10, 20, 30],
 "json": {
 "key": null
 }
}

Jason - The Parser

Meet Jason! It is the name of our JSON parser. We will use Python to create it. So let’s begin with creating a file called jason.py and call it a day! Please don’t name it json.py as it conflicts with Python’s standard module json.

Our parser will consist of tiny functions, each meant to extract some value. After parsing, we get a Python object that matches the data we parsed.

#	JSON Type	Python Type
1.	List	`list`
2.	JSON	`dict`
3.	Number	`float` or `int`
4.	String	`str`
5.	`null`	`None`

The Preparation

Our data is a string, that is text. It can come from two places, either a file or a Python str. We won’t be reading the data in an entire go, we will read it character by character. This presents some problems:

We need a short and standard API to read both string and file. Python strings are a sequence. It means, we can access individual characters by their indices.
```
name = "Joe"
print(name[1]) # o
```
This syntax is short, just two square brackets and a number inside it.
But, when we try to read an index beyond the string’s length, we get an IndexError.
```
name = "Joe"
print(name[100]) # IndexError: string index out of range
```
Wrapping every index access inside a try except block is too verbose. So we need a workaround for this noise.

For the first problem, we will create a reader that takes care of taking an index and returning a character. For the second problem, we will create a special class EOF (end of file), which is returned whenever we try to read beyond the length.

This is how our EOF looks like:

class EOF:
 pass

The abstract class (or template) for our reader is:

class Data:
 def __getitem__(self, i: int):
 raise NotImplementedError

Here __getitem__ is a magic method. This allows us to access the data at index via data[i] syntax. Our Data class is abstract, that’s why it raises NotImplementedError.

Let’s write the concrete classes now. For a string data:

class StringData(Data):
 def __init__(self, txt: str):
 self._txt = txt

 def __getitem__(self, i: int):
 try:
 return self._txt[i]
 except:
 return EOF

We store the data in a private variable _txt. See how our __getitem__ has been overwritten with new code. We wrap the index access inside a try except and return EOF if we get any error.

For data from file:

from typing import IO


class FileData(Data):
 def __init__(self, fp: IO):
 self._fp = fp

 def __getitem__(self, i: int):
 self._fp.seek(i, 0)
 char = self._fp.read(1)
 if char == "":
 return EOF
 return char

At the first line, we import IO from typing to set type hint for our file object. Our class takes a file pointer and stores it in _fp. To get a character at an index, we first seek to that index via fp.seek. The first argument i is the position and the second argument 0 means the position i is counted from the beginning of the file.

We have to do seeking because, Python automatically moves our file cursor when we read the data. So two consecutive read calls to the file gives the subsequent data stored in it. Also, unlike str, trying to read a file beyond its length returns an empty string. To be consistent with our reader API, we check for an empty string and return EOF in place of it.

Putting this all together, our jason.py should look like this:

from typing import IO


class EOF:
 pass


class Data:
 def __getitem__(self, i: int):
 raise NotImplementedError


class StringData(Data):
 def __init__(self, txt: str):
 self._txt = txt

 def __getitem__(self, i: int):
 try:
 return self._txt[i]
 except:
 return EOF


class FileData(Data):
 def __init__(self, fp: IO):
 self._fp = fp

 def __getitem__(self, i: int):
 self._fp.seek(i, 0)
 char = self._fp.read(1)
 if char == "":
 return EOF
 return char

Let’s do a little testing of our API. You need to launch an interpreter and load the module in it. Assuming the module is in the same location as your terminal, you can simply do python -i jason.py. (On some systems it may be python3 or py or py3).

~/Scratch ❯ python3 -i jason.py
>>> string = "Hello World!"
>>> data = StringData(string)
>>> data[0]
'H'
>>> data[5]
' '
>>> data[7]
'o'
>>> data[2]
'l'
>>> data[100] # beyond length
<class '__main__.EOF'>
>>> data[100] == EOF
True
>>> fp = open("test.json", "w") # prepare a sample
>>> fp.write('"Hello"') # write data in it
7
>>> fp.close() # save the sample
>>> fp = open("test.json") # open it back for reading
>>> data = FileData(fp)
>>> data[0]
'"'
>>> data[1]
'H'
>>> data[2]
'e'
>>> data[10]
<class '__main__.EOF'>
>>> data[10] == EOF
True

Yay! Our reader is ready. Now we are moving to the core, where we extract the values. 🐕

Defining Constants

Our syntax has a few literals like comma, colon, quotes etc. Let us define them as constants.

COMMA = ","

JSON_START = "{"
JSON_END = "}"
JSON_SEP = ":"

LIST_START = "["
LIST_END = "]"

PERIOD = "."
MINUS = "-"
PLUS = "+"

NULL = "null"

STRING_START = STRING_END = '"'

TERMINALS = (EOF, COMMA, LIST_END, JSON_END)

Most of the constants should be self explanatory. TERMINALS is a tuple with literals which are used to indicate the end of a value. For example, when parsing a data like 123,, we know the number stops at comma, hence it is a terminal.

Our Functions

We will write separate functions to extract different values. Each of these functions take two arguments - a Data data and an integer pos. It means that the function has to read data from position pos and extract the value from it.

These functions read till they meet a terminal character. Then, they convert the read string to valid data type and return it along with the position they have read so far. This position is used by the next function to know from where the next value should be read.

It could be a little confusing, but once you see it in practice, I hope it will be clear.

Extracting Value

JSON can be parsed recursively. We start with making an entrypoint function that matches the first character and calls the appropriate extraction function.

def extract_value(data: Data, pos: int):
 char = data[pos]

 if char == EOF:
 return "", pos

 if char == JSON_START:
 val, pos = extract_json(data, pos)
 elif char == LIST_START:
 val, pos = extract_list(data, pos)
 elif NULL.startswith(char):
 val, pos = extract_null(data, pos)
 elif char.isdigit() or char in (MINUS, PLUS):
 val, pos = extract_number(data, pos)
 elif char == STRING_START:
 val, pos = extract_string(data, pos)
 else:
 raise ValueError(f"unexpected character: {repr(char)}")

 return val, pos

The first if checks whether the data has any character left. If we get EOF, it means either the data is empty or that data has been already read. In that case, we return with an empty string and an unmodified position.

The subsequent ifs match the character with a known data type and on success, they call the respective extract function. At last the else is there as fallback, which raises a ValueError, because that character doesn’t match our syntax guide.

repr function is used to print the character’s representation. We use repr instead of directly printing the character to help in case of non-printable characters.

>>> char = "\t"
>>> print(f"Oops what character is this {char}")
Oops what character is this
>>> print(f"Oops what character is this {repr(char)}")
Oops what character is this '\t'

See how the first print prints a tab which is quite confusing as it is not visible. However the second print clearly prints the escape character.

Skipping Space

As said earlier, we are going to be make space insignificant in our syntax. So any space character, (unless they are within quotes) is gently ignored. We now create a function that does the same.

def skip_space(data: Data, pos: int):
 char = data[pos]
 while char != EOF and char.isspace():
 pos += 1
 char = data[pos]

 return pos

This function reads the data character by character and stops only when it encounters EOF or a non-space character. After the encounter, it returns the position of the non-space character.

>>> skip_space(StringData(" 1"), 0)
4
>>> skip_space(StringData(" 1 "), 0)
4
>>> skip_space(StringData("\t\n X"), 0)
3

Our function cares only about the leading space.

Extracting Number

The logic for extracting a number is as follows:

If the number starts with plus, we ignore it. For minus, we add it to our string buffer.
We then skip the space (continue reading to know why).
We use is_float as flag to know if the number is a float.
Characters are read from the data as long as they are not a terminal.
If the character is a digit, we add it to our buffer.
If the character is a period (.), we check if is_float is True.
If it is True, it means we already found a period in the number. It is an error, so we raise an exception.
In case, is_float is False, we set it to True and append the period to our buffer.
If the character is a space, we skip all of them. After skipping, we check if the non-space character is a terminal, if not we raise an exception.
The last else raises an exception as the character is neither a digit nor period nor space.
Finally, we convert the buffer to a float or an integer depending upon the value of is_float.

So the function looks like this:

def extract_number(data: Data, pos: int):
 if data[pos] in (MINUS, PLUS):
 num = "-" if data[pos] == MINUS else ""
 pos += 1
 pos = skip_space(data, pos)
 else:
 num = ""

 is_float = False
 char = data[pos]

 while char not in TERMINALS:
 if char.isdigit():
 num += char
 pos += 1
 char = data[pos]
 elif char == PERIOD:
 if is_float:
 raise ValueError("extra '.' found in floating point number")
 is_float = True
 num += PERIOD
 pos += 1
 char = data[pos]
 elif char.isspace():
 pos = skip_space(data, pos)
 char = data[pos]
 if char not in TERMINALS:
 raise ValueError("invalid syntax for number")
 else:
 raise ValueError(f"unexpected character in number: {repr(char)}")

 if is_float:
 return float(num), pos
 else:
 return int(num), pos

First of all, we allow space after the signs just for the visual sake. This allows us to create lists like this:

[
+ 10,
- 15,
+ 3,
+ 5,
]

Not really that great, but yea, it exists.

We don’t allow multiple periods to prevent numbers like 12.34.56, which is not valid at all. Similarly, we check the first character after skipping space to terminals. For example consider the data 12.34 13. What does that even mean? If you think, it should be parsed as 12.3413, then what about data like 12.34 "hello" or 12.34 [10, 20]? To avoid any of such ambiguity, we don’t want any non-terminal character to follow a number.

Now let us check our function with some values.

>>> extract_number(StringData('12'), 0)
(12, 2)
>>> extract_number(StringData('10.0'), 0)
(10.0, 4)
>>> extract_number(StringData('-2.71'), 0)
(-2.71, 5)
>>> extract_number(StringData('- 3.14 '), 0)
(-3.14, 10)
>>> extract_number(StringData('10.20.30'), 0)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/arun-mani-j/Scratch/jason.py", line 172, in extract_number
 raise ValueError("extra '.' found in floating point number")
ValueError: extra '.' found in floating point number
>>> extract_number(StringData('10.0 20'), 0)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/arun-mani-j/Scratch/jason.py", line 181, in extract_number
 raise ValueError("invalid syntax for number")
ValueError: invalid syntax for number
>>> extract_number(StringData('Not a number'), 0)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/arun-mani-j/Scratch/jason.py", line 183, in extract_number
 raise ValueError(f"unexpected character in number: {repr(char)}")
ValueError: unexpected character in number: 'N'

Extracting String

Compared to numbers, strings are easy. We start the double quotes and append every character we get to the buffer till we meet either a double quote or EOF.

def extract_string(data: Data, pos: int):
 string = ""
 pos += 1
 char = data[pos]

 while char not in (STRING_END, EOF):
 string += char
 pos += 1
 char = data[pos]

 if char == EOF:
 raise ValueError(f"string terminated without: {STRING_END}")

 pos += 1
 pos = skip_space(data, pos)

 return string, pos

The first pos += 1 skips the double quotes character. From there we read till we get EOF or double quotes. Once our while loop completes, we check if the last character is EOF. If yes, it means that our string has not been properly closed with a double quote, so we raise an exception.

Otherwise, we do another pos += 1 to go past the quotes. Then we skip the space following the string and then return the extracted value.

Let’s test our function to make sure it works well.

>>> extract_string(StringData('"Hello, World!"'), 0)
('Hello, World!', 15)
>>> extract_string(StringData('"Hello, World!" '), 0)
('Hello, World!', 24)
>>> extract_string(StringData('"Hello, World! " '), 0)
('Hello, World! ', 27)
>>> extract_string(StringData('"New \n Line"'), 0)
('New \n Line', 12)
>>> extract_string(StringData('"Not closed'), 0)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/arun-mani-j/Scratch/jason.py", line 202, in extract_string
 raise ValueError(f"string terminated without: {STRING_END}")
ValueError: string terminated without: "

Extracting List

Similar to strings, lists are enclosed inside square brackets. But in addition, they contain commas inside. Since our functions are already made with terminals in mind, we just call extract_value whenever we find a character that is not a comma or space.

def extract_list(data: Data, pos: int):
 list = []
 pos += 1
 char = data[pos]
 found = False

 while char not in (LIST_END, EOF):
 if char == COMMA:
 if not found:
 raise ValueError("empty comma in array")
 found = False
 pos += 1
 elif char.isspace():
 pos = skip_space(data, pos)
 else:
 val, pos = extract_value(data, pos)
 list.append(val)
 found = True

 char = data[pos]

 if char == EOF:
 raise ValueError(f"list terminated without {LIST_END}")

 pos += 1
 pos = skip_space(data, pos)

 return list, pos

The flag found is used to avoid empty commas like [1, ,, 20]. We set it to True when we append some value, so that the function knows that the comma was indeed used with a value before it. When we meet a comma, we again reset it to False.

We call skip_space when we find a space as spaces can be ignored inside a list.

Time to check our function with some values.

>>> extract_list(StringData('[1, 2, 3]'), 0)
([1, 2, 3], 9)
>>> extract_list(StringData('[]'), 0)
([], 2)
>>> extract_list(StringData('[ ]'), 0)
([], 4)
>>> extract_list(StringData('["Hello", 12.3, -3, [1, 2 ]]'), 0)
(['Hello', 12.3, -3, [1, 2]], 28)
>>> extract_list(StringData('[1, 2,, 4]'), 0)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/arun-mani-j/Scratch/jason.py", line 115, in extract_list
 raise ValueError("empty comma in array")
ValueError: empty comma in array
>>> extract_list(StringData('[1, 2'), 0)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/arun-mani-j/Scratch/jason.py", line 128, in extract_list
 raise ValueError(f"list terminated without {LIST_END}")
ValueError: list terminated without ]

Extracting JSON

Luckily, extracting JSON is as same as lists but with an addition of extracting both key and value. A key-value is separated by a colon and there can be any amount of space between them. As we mentioned in our spec, keys can be of only string type. So anything else as a key raises an exception.

def extract_json(data: Data, pos: int):
 json = {}
 pos += 1
 char = data[pos]
 found = False

 while char not in (JSON_END, EOF):
 if char == COMMA:
 if not found:
 raise ValueError("empty comma in JSON")
 found = False
 pos += 1
 elif char == STRING_START:
 key, pos = extract_key(data, pos)
 val, pos = extract_value(data, pos)
 json[key] = val
 found = True
 elif char.isspace():
 pos = skip_space(data, pos)
 else:
 raise ValueError("expected key")

 char = data[pos]

 if char == EOF:
 raise ValueError(f"JSON terminated without {JSON_END}")

 pos += 1
 pos = skip_space(data, pos)

 return json, pos

Here we check for double quotes and if found, we extract the key from it. And then we extract the value. The extract_key function is given below.

def extract_key(data: Data, pos: int):
 key, pos = extract_string(data, pos)
 char = data[pos]

 if char != JSON_SEP:
 raise ValueError(f"expected {JSON_SEP} but found: {repr(char)}")

 pos += 1
 pos = skip_space(data, pos)

 return key, pos

The logic to extract a key is to first extract the string. Then we check if the string is followed by a colon. If not, we raise an exception. Remember that extract_string already skips the spaces following the quotes, so data like {"hello": "space before me"} is extracted properly.

Let’s now test these functions.

>>> extract_json(StringData('{"key": 10,}'), 0)
({'key': 10}, 12)
>>> extract_json(StringData('{ "key" : 10 ,}'), 0)
({'key': 10}, 24)
>>> extract_json(StringData('{}'), 0)
({}, 2)
>>> extract_json(StringData('{"key 1": 10, "key 2": { "key 3": 20}}'), 0)
({'key 1': 10, 'key 2': {'key 3': 20}}, 38)
>>> extract_json(StringData('{10: "key not string"}'), 0)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/arun-mani-j/Scratch/jason.py", line 80, in extract_json
 raise ValueError("expected key")
ValueError: expected key
>>> extract_json(StringData('{"key" = "not colon"}'), 0)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/arun-mani-j/Scratch/jason.py", line 73, in extract_json
 key, pos = extract_key(data, pos)
 ^^^^^^^^^^^^^^^^^^^^^^
 File "/home/arun-mani-j/Scratch/jason.py", line 98, in extract_key
 raise ValueError(f"expected {JSON_SEP} but found: {repr(char)}")
ValueError: expected : but found: '='

Extracting `null`

Finally, let’s make the billion dollar mistake by extracting null. To do so, we read four characters and check they equal null. We also ensure that the read character is not a terminal so we can quit in advance. For example, if our data is nu,2, then once we find comma, there is no point in reading the next character (2).

def extract_null(data: Data, pos: int):
 word = ""
 char = data[pos]
 count = 0

 while char not in TERMINALS and count != len(NULL):
 word += char
 count += 1
 pos += 1
 char = data[pos]

 if count == len(NULL) and word == NULL:
 pos = skip_space(data, pos)
 return None, pos

 raise ValueError(f"unexpected literal: {repr(word)}")

We actually used len(NULL) instead of four because we don’t want to hardcode the value as changing NULL means the length should also be updated.

Now some testing of the function.

>>> extract_null(StringData('null'), 0)
(None, 4)
>>> extract_null(StringData('null '), 0)
(None, 6)
>>> extract_null(StringData('nu,0'), 0)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/arun-mani-j/Scratch/jason.py", line 151, in extract_null
 raise ValueError(f"unexpected literal: {repr(word)}")
ValueError: unexpected literal: 'nu'

Are We Done Yet?

This is getting a bit lengthier than it should be… Okay, so now our function extract_value is ready, we should be able to extract JSON pretty fine… right? Well yes but,

In the tests we conducted so far, we manually passed a StringData and position. To be a good library, we should provide some shortcuts.
extract_value just extracts a value. Yes, only one value. It means, if your data is "hello" "they do not know me", then our function parses only "hello".
```
>>> extract_value(StringData('"hello" "they do not know me"'), 0)
('hello', 9)
```
But the given data is invalid.

Let us introduce three more functions to tackle the above problems.

def extract(data: Data):
 val, pos = extract_value(data, 0)
 char = data[pos]

 if char != EOF:
 raise ValueError(f"unexpected character: {repr(char)}")

 return val


def load(fp: IO):
 return extract(FileData(fp))


def loads(txt: str):
 return extract(StringData(txt))

The first function extract takes a Data as argument and extracts the value in it, AND, it checks if the character left after the value is EOF. If it is not EOF, it means some other characters are still there, which is an invalid syntax. So, we raise an exception. Since, our extract_* functions already consume space, the character left must be either EOF or a non-space character.

The last two functions load and loads are similar to json.load and json.loads respectively.

load takes a file pointer and parses the data from it. loads parses the data from a Python str.

Are We Done Yet??!!

Yes yes, we are! Now let us test our parser with a sample JSON data.

{
 "hello": 1,
 "another": +2,
 "extra": -3,
 "then": 2.17,
 "but": [1, 2, 3],
 "also": { "1": null, "2": [] },
 "end": "Bye"
}

Let’s say this file is saved as test.json. Now let us call our parser.

~/Scratch ❯ python3 -i jason.py
>>> fp = open("test.json")
>>> json = load(fp)
>>> fp.close()
>>> from pprint import pprint
>>> pprint(json)
{'also': {'1': None, '2': []},
 'another': 2,
 'but': [1, 2, 3],
 'end': 'Bye',
 'extra': -3,
 'hello': 1,
 'then': 2.17}

Oh yea, it works 🙂. We used pprint to pretty print the returned dictionary. Let us now check what happens if we pass more than one value as data.

>>> loads('"parse me" "not me"')
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/arun-mani-j/Scratch/jason.py", line 247, in loads
 return extract(StringData(txt))
 ^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/arun-mani-j/Scratch/jason.py", line 237, in extract
 raise ValueError(f"unexpected character: {repr(char)}")
ValueError: unexpected character: '"'
>>> loads('"parse me" 3.14')
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/arun-mani-j/Scratch/jason.py", line 247, in loads
 return extract(StringData(txt))
 ^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/arun-mani-j/Scratch/jason.py", line 237, in extract
 raise ValueError(f"unexpected character: {repr(char)}")
ValueError: unexpected character: '3'

Yes! It fails successfully 🎉.

Conclusion

We managed to write a parser for our JSON-like syntax perfectly well. Of course, it can be improved a lot. May be the entire logic is flawed and we should rewrite with a different algorithm. However, it is beyond our scope and objective. We wanted a silly parser and now we got it.

The full source code of jason.py in my snippets.

Thanks for reading! Please feel free to share your thoughts and suggestions. You can contact me via the links in footer.

Good day! 👐

Aayra

Arun Mani J (j.arunmani@proton.me) — Mon, 02 Jan 2023 00:50:24 +0530


Language	Python
Repository	https://gitlab.com/arun-mani-j/aayra
GUI Framework	GTK, Adwaita

Aayra is an eBook reader and audiobooks player. Its speciality is being able to do both of the functionality within the same app in a simplistic manner.

It uses GTK for GUI and SQLite for database. The app is under heavy development, so you can keep track of the repository to know the changes.

aurthy

Arun Mani J (j.arunmani@proton.me) — Mon, 02 Jan 2023 00:50:16 +0530


Languages	Python, TypeScript
Frontend	Nuxt, TailwindCSS, daisyUI
Backend	FastAPI
Database	PostgreSQL
Repository	https://gitlab.com/aurcc/aurthy

aurthy is a university project that lets a person or entity sign and distribute documents digitally. The receiving person can verify the originality by using the unique identifier. The aim of the project was to reduce the paperwork and go completely digital for everything from certificates to permission forms etc.

The working principle behind aurthy is simple. Each user gets a username and password. After logging in to their account, they can upload the documents and provide a username of the signer. The signer then receives the request, where they verify the credentials and approve the request. This generates a unique identifier using UUID and a certificate with QR code in it, which is then sent to the requester. Using the certificate, anybody can verify the authentication by scanning the QR code.

The threat model here is that the server is safe and if the server is able to verify a certificate then the certificate is valid.

The backend was written in Python using FastAPI framework. Frontend uses Nuxt with TailwindCSS and daisyUI for CSS. The data regarding users and documents were stored in a PostgreSQL database.

The project is currently in a halt because the HR team of the college said that many firms do not accept digital signatures. Plans are there to expand the scope of digital certificates using government schemes.

Aashii

Arun Mani J (j.arunmani@proton.me) — Mon, 02 Jan 2023 00:50:11 +0530


Language	Python
Repository	https://github.com/arun-mani-j/Aashii

Aashii is a Telegram bot to simplify communication between members and admins of a group. Consider a Telegram group a lot of members and multiple admins. Here, the ability to let every user personally message an admin for any issue is a big trouble, both in terms of privacy and availability. To simplify this communication, Aashii is used.

Initially, all the admins of a group are added to a separate private group with Aashii as a member. Now, any member who wishes to contact the admins can contact the bot. The bot acts as a middle-hand forwarding messages in both directions.

In addition to message forwarding, Aashii supports multiple features like flood resistance, on the fly edits of messages, complete media support and broadcasting.

The bot is written in Python using python-telegram-bot framework and PostgreSQL is used as the database management system.

Aashii is a self-hosted solution. It is currently being used by many Telegram users to provide customer service to their members.

DoNotRepeatBot

Arun Mani J (j.arunmani@proton.me) — Mon, 02 Jan 2023 00:49:54 +0530


Language	Python
Repository	https://github.com/arun-mani-j/DoNotRepeatBot

DNR is a Telegram bot that allows you to share messages via inline feature of Telegram. By storing repetitive messages in the bot, you can send the messages to any chat by calling the bot inline.

Initially, the bot worked only for text messages (so called snippets) but as days progressed, I rewrote the bot to allow any media type that Telegram allows to be sent via inline query. This means photos, audios, videos, documents and of course texts.

The version 2.0 also saw a new profile picture, designed by a Telegram user named Happy Princess, for the bot.

Personally I used to use this bot a lot to send messages in GNU/Linux groups where few questions liking mixing desktop environments, clarifications on WSL and normal distro are frequent.

The bot is written in Python using python-telegram-bot framework and PostgreSQL is used as the database management system.

DNR was hosted on Heroku and available to users via @donotrepeatbot bot account, but after Heroku decided to drop its free plan, the bot is inactive. However, DNR was a fun ride as it was the first Telegram bot I made which used inline queries.

Update The bot is back alive thanks to Suhasa SJ!

Debian Day’22, Bishop Heber College, Trichy

Arun Mani J (j.arunmani@proton.me) — Wed, 24 Aug 2022 18:30:00 +0530

Debian GNU/Linux, the universal operating system was officially released in 16 August 1993 by Ian Murdock. To celebrate the birth day of Debian, open source community and enthusiasts gather together, which is known by the name Debian Day. Anyone is free to organize the event both virtually and physically. The main objective of the Day is celebration, so any strict rules and protocols are relaxed.

The free and open source users of south India planned to conduct the event in Kerala and Tamil Nadu. Due to unfortunate flood related issues in Kerala, the event was done through online. In Tamil Nadu, we organized the Day in Bishop Heber College of Trichy. From the entire making blueprint to bringing it in action was done by the students of BHC led by Ravishanker. Ash and Abraham Raji guided us in the process. I reached the spot with the help of Ravi on 16 August.

The event was to be simple but fun. Since it is organized in a college where it is highly possible that many are not aware of what Debian or Linux is, we shifted our focus from just celebrating Debian to celebrating by spreading awareness. Or in the other words, we wanted students and the audience to know what Debian is, about the free and open source software etc.

Vice Principal of BHC presenting the welcome speech

The Day with prayer song followed by welcome speech by the Vice Principal of BHC. He was such a lively person whose mere presence itself meant a big support. He said he spent a few hours last night on searching about Debian, dpkg etc. He was very happy to know that the event is organized by students who are involved in free and open source contributions during their studies itself. We then joined with the Principal of BHC who met us virtually and welcomed us to the event.

After his speech, Ravishanker presented a lightning talk on FOSS. He had a stuffed up presentation with memes and points that was successful in engaging the audience to the stage.

Trishiraj on the day

Following that, Trishiraj, the lead developer of CorvusOS joined us via online. Being a native of Tamil Nadu, he spoke in a mix of Tamil and English, which was highly useful because the audience felt his words. Trishi talked about his ROM, the Android Open Source Project and a layperson guide to contributions.

After this, I presented a speech on history of Debian to how to get started with it. I also included my personal history on how I got familiar with both FOSS and Debian.

Kyle presenting his talk

We had a small tea-break which was followed by a talk by Kyle Robbertze, Debian Developer. Kyle joined us virtually and presented about the advantages of Debian, the contribution guide and etc. Kyle’s talk brought a lot of questions like aboutpackage management, Debian’s Salsa etc. Due to technical issues on our side, I attended the QA for him. The final event was an interactive quiz on FOSS. The questions were based on different spectrum of FOSS but from the user’s perspective. The last question was a Beat the Clock, with the question of naming as many as FOSS applications/projects as possible. We gave a small memento to the student who named the majority.

A few organizers of the event, Ravi on the extreme right and me to his left

The Day was awesome. It was the first event we conducted both on our own and represented the Debian community (instead of students). The main target we set for the event was that we need to kindle a spark in the minds of students and staffs about Debian and FOSS. In that regard, the event is a grand success. We had the chance to meet the Vice Principal after the event and he was very happy for the Day. He was so inspired by Debian and the community that he said that they will and be welcoming to conduct more such events.

The Day could not have been possible without the FOSS community and so, it is neither you, nor me, it is us. We did it. Looking forward to join more such events in the coming days!

One Hundred Years of Solitude

Arun Mani J (j.arunmani@proton.me) — Mon, 02 May 2022 14:10:45 +0530

A brilliant narration of a family.

The Pregnant King

Arun Mani J (j.arunmani@proton.me) — Mon, 02 May 2022 14:09:51 +0530

A wonderful combination of magic and philosophy.

Educated

Arun Mani J (j.arunmani@proton.me) — Mon, 07 Mar 2022 13:43:10 +0530

Power of education for someone from a conservative family.

Norwegian Wood

Arun Mani J (j.arunmani@proton.me) — Tue, 08 Feb 2022 13:42:37 +0530

A big love tale.

Quack

Arun Mani J (j.arunmani@proton.me) — Thu, 03 Feb 2022 20:28:08 +0530


Language	Rust
Repository	https://gitlab.com/arun-mani-j/quack

Quack is a command-line app to search the web via DuckDuckGo. It uses DDG’s API to fetch the instant answers and results.

The project is very simple in nature with the main objective to learn and try out Rust for something realistic. The app uses reqwest for the HTTP requests, clap for command-line handling and serde_json to parse JSON.

New Website

Arun Mani J (j.arunmani@proton.me) — Thu, 03 Feb 2022 00:49:24 +0530

This is the first blog post I’m publishing in this website. I just finished adding final touch-ups to the site, which took me a three long evenings to make. With the finish, I thought of removing the placeholder posts (if you want to see those placeholders, then you can look at the source code repository 😉) and adding real articles. Yes, this is a real article.

Preface

To get started, I’m not a professional website developer, which you might have guessed with how the site looks 🙈, but still, I thought of giving it a try. I’m not really a fan of making websites because

They are really difficult
I’m not that creative
I don’t have enough resources

The above statements might sound vague to you, so let me explain.

They are really difficult

Yes. Gone the days with simple websites. Nowadays every website has a pretty complexity around it. From a rich collection of frameworks to a massive list of variations like browsers (Chrom*, Firefox, Safari etc.), screen sizes (desktop, phone, tablet etc.), personal preferences (dark mode, privacy conscious etc.). A professional website developer may not be worried of these, but as I said in the starting, I’m not a professional. So, it is not really easy from a newbie point of view.

I’m not that creative

Creativity and me are opposite ends. I’m not really a fan of eyes-stealing fancy frills. I like to keep things simple, easy and tweak-able. Apart from that, I’m a sort of person who feels that I should not spend too much time in something that I don’t understand quite well. An example is navigation menu. I wanted to avoid JavaScript at the client side, so I looked for alternatives. One such, I found is a hack using checkbox. I tried it, it neither worked nor I understood why. I left it.

I don’t have enough resources

This is a big obstacle in my tech life and so it is for web development. I have a laptop with 2 GB RAM. Yep very old one. You know how difficult it is even to browse modern day websites, then how can I even dream of making one. Thanks to low-memory environment I have made myself comfortable of, I was able to make this site finely. The low-memory environment is nothing but SwayWM and GNU Emacs. I will explain about it later.

Now you might get an idea on the problems I face. Yet, I was able to finish the site by doing this and that. Oh yes!

My Old Website

The first real world website I made is a website for me. I was a very newbie to web development that time. I wanted a website for namesake. So I used (still use) GitLab Pages for hosting. I wrote the code using VueJS and it was a very fine mess. But it did work.

It was not made for blog or ideas sharing, but just a website for namesake. The site was not colorful, it was full black text on a clear white background.

Soon after making the site, I forgot to improve it and later forgot its very existence.

Bedtime Inspiration

Thanks to the COVID-19 pandemic, I got a lot of free time. In that free time, I was able to meet new people of different point of views (online and in Telegram of course!). From them, I learnt of new technologies and frameworks. But apart from those frameworks, I learnt about simple awesome websites. There are a lot to name. Those are the websites that don’t use JavaScript or try to nuke your privacy with ads and data harvest.

Examples include websites of GNU, Debian, Drew DeVault. They were simply fascinating, and that’s how I was also inspired to make a website that is simple. That’s it.

Time To Start Playing

So now it’s time to start making the website. I decided to use Hugo for building the website and TailwindCSS for CSS. Now after the finish of site, I think I have made a solid decision to use them. They have good documentation and a pretty big community. It really helped.

Also thanks to a vast resources and tutorials available in Internet guiding various topics like semantic web, image optimization etc.

Finally, I solved the first two issues I mentioned earlier, what about the third issue?

First workspace with Firefox open.

I used to keep three workspaces in Sway. The first has Firefox with two tabs. The first tab has localhost:1313 with my developing website. The second tab has documentation of either Hugo or TailwindCSS or any important site.

Second workspace with Emacs open.

The second workspace has Emacs with code of my site. I don’t know how many buffers will be active there, most of the time it is around ten. But at the front (means at Emacs window), I keep only one or two panes at most (two panes if I’m referencing some other code).

Third workspace with Foot windows open.

The third workspace has two Foot terminal windows. Since the website is in development stage, I run watch mode of TailwindCSS and Hugo in the two windows. Watch mode is nothing but a watchdog that reloads and builds the website on file changes.

With this setup, my memory usage comes around 80%. And sometimes I need to debug the site, so I have to open Firefox’s developer tools. So I’m pretty choking my system. But, this choking is the main reason, I need the website to be minimal. Neither can I make a website that is greedy on resources without killing my laptop 🤷.

Conclusion

After pretty good headache, I finally finished the site. Also, for your kind information, I didn’t jump directly from my old VueJS based site to this one. In between, I spent a lot of time in making 2 to 3 websites for my friends and contacts. For them, I used Hugo and TailwindCSS. This strengthened my experience and helped me in making my site without a lot of hassle.

My first blog post should not be so long, so let’s finish it up. The website’s source code is available for public reference in my GitLab repository, feel free to have a look. Don’t forget to share your feedback and thoughts via any Telegram or EMail (links are there in footer!).

Thanks for your time 😊.

New Update

Time has changed, I bought a new laptop - ThinkPad E14. Now, the memory constraints are pretty much not an issue. I also learnt about daisyUI and rewrote the website with a few parts using it. A lot of hard coded values were replaced with the help of daisyUI style classes. I’m now using GNOME but GNU (Doom) Emacs is still my default text editor. Somethings don’t change right 😉?

A Man Called Ove

Arun Mani J (j.arunmani@proton.me) — Fri, 07 Jan 2022 13:41:41 +0530

Ove is the chad neighbor we all hate but at the end miss.

The Jaunt

Arun Mani J (j.arunmani@proton.me) — Fri, 19 Nov 2021 13:41:19 +0530

A future that is scary because of science.

The Little Prince

Arun Mani J (j.arunmani@proton.me) — Sat, 02 Oct 2021 14:10:12 +0530

A refreshing story that was fun to read.

The Husband Stitch

Arun Mani J (j.arunmani@proton.me) — Sat, 02 Oct 2021 11:53:47 +0530

A scary, interesting short story.

The Handmaid's Tale

Arun Mani J (j.arunmani@proton.me) — Thu, 02 Sep 2021 11:51:32 +0530

If this is the future, then please don’t.

Hyperbole and a Half

Arun Mani J (j.arunmani@proton.me) — Mon, 02 Aug 2021 11:52:32 +0530

Beautiful narration of various scenarios.

Anxious People

Arun Mani J (j.arunmani@proton.me) — Mon, 02 Aug 2021 11:51:19 +0530

A refreshing tale of our human lives.

Forever Series

Arun Mani J (j.arunmani@proton.me) — Fri, 02 Jul 2021 14:09:21 +0530

A modern love tale with suspense and mystery.

In a Grove

Arun Mani J (j.arunmani@proton.me) — Fri, 02 Jul 2021 11:53:15 +0530

Awesome mystery surrounding a murder.

Strange Planet

Arun Mani J (j.arunmani@proton.me) — Fri, 02 Jul 2021 11:51:09 +0530

After reading, you realize that Earth is really strange.

The Song of Achilles

Arun Mani J (j.arunmani@proton.me) — Wed, 02 Jun 2021 11:50:58 +0530

Trojan War but from a different amazing viewpoint.

My Days in the Underworld

Arun Mani J (j.arunmani@proton.me) — Fri, 02 Apr 2021 11:52:06 +0530

From the man who ruled the underworld.

Sherlock Holmes

Arun Mani J (j.arunmani@proton.me) — Tue, 02 Mar 2021 14:09:27 +0530

Each story teaches you a new form of out-of-box thinking.

The Kite Runner

Arun Mani J (j.arunmani@proton.me) — Mon, 02 Nov 2020 11:50:35 +0530

Beautiful narration of a culture, traditions and difficulties.

Man's Search for Meaning

Arun Mani J (j.arunmani@proton.me) — Wed, 02 Sep 2020 11:50:22 +0530

A lot of tough and difficult scenarios.

Crime and Punishment

Arun Mani J (j.arunmani@proton.me) — Sun, 02 Aug 2020 11:49:55 +0530

I went full upside down reading this book. Perspective changing story.

The Girl on the Train

Arun Mani J (j.arunmani@proton.me) — Thu, 02 Jul 2020 01:01:12 +0530

Fantastic thriller with a nice twist at the end.

Recursion

Arun Mani J (j.arunmani@proton.me) — Tue, 02 Jun 2020 00:53:32 +0530

First ever sci-fi novel I have ever read. As always time machines are dangerous.

Nothing Lasts Forever

Arun Mani J (j.arunmani@proton.me) — Mon, 20 Jan 2020 13:42:18 +0530

Mysteries within mysteries.

Arun Mani J (j.arunmani@proton.me) — Mon, 01 Jan 0001 00:00:00 +0000

About Me

Note: If you need a formal description, then please check out my resume.

I’m Arun Mani. You might me find me online with names J Arun Mani or Arun Mani J. Here J is my initial. Please address me by my full name Arun Mani or just Arun. Whatever you feel comfortable with 🙂.

My native is Tamil Nadu, India. I can speak Tamil (mother tongue!), English and can understand and speak basic Hindi.

I’m currently studying B.E Computer Science Engineering in Anna University, Regional Campus, Coimbatore. Apart from studies, I spend time reading books, coding and reading articles.

Computer Science

I can’t find a better name for this section as I think I’m not just a programmer. I mean, yes I’m one but not just one. 😉

I started coding at grade 9 (9th standard in Indian terms). For me, I was attracted to computers by video games. After a lot of web searches, I started with Python. Now after a few years, I know Python quite well. I also know basics of C, Java, JavaScript and Rust. My personal favorite is Python and Rust, because Python is simple and elegant and Rust has such a lovely compiler.

Free Libre and Open Source

I’m an advocate of free libre and open source (FLOSS) technologies with respect for privacy. Most of the software I used are based on this opinion. But there are some things, both proprietary and privacy destroying, that I have to use because of various social constraints.

Nerdy Cool Things

I use Doom Emacs, a configuration kit for GNU Emacs. For operating system, I use GNU Debian Linux. Simplicity is key!

Reading Preferences

I like books on fiction. Non-fiction is fine as long as it is not boring. I also love reading books on philosophy 🌟. I’m not an avid reader, but I try my best to complete at least one book a month. If you got a good book to share, don’t forget me!

Reading blogs and articles is also my way of passing time. I try to read anything that is mildly interesting.

Contact Me

Feel free to contact me for any feedback, comments and chit chat. For instant reply, use Telegram, for classic way, use email. The links are available in footer.

Arun Mani J (j.arunmani@proton.me) — Mon, 01 Jan 0001 00:00:00 +0000

Really Simple Syndication (RSS)

RSS stands for Really Simple Syndication. If you need to dive deep into the specification, then you should check out the official website. Here, I’m going to focus on the user’s perspective. Please note that the article is not fool-proof, some mistakes surely exist. Please let me know, so I can update it.

What is RSS?

RSS is a format by which you can access the updates to a website. The website developer, stores the changes in a file, for example in www.example.org/index.xml, the index.xml contains the changes made to the website. You on your end, will use an app known as news aggregator to access the updates. So all you need to do is, install an aggregator app, add the website to its list. Now that app will let you know of the changes whenever it happens. The only requirement is that the developer of website has RSS enabled. Sounds simple right?

Many modern websites actually use newsletter, where you provide your email and they send you all the updates. Let’s see what are its advantages and disadvantages.

Advantages

Emails can be personalized. The company can target your needs and draft the content in a way you might like. RSS on the other hand is plain and same for everyone. Though, some tweaks can be done, that is a headache.
Delivery of an email is ensured (provided the email address is valid), which means the company knows that you actually received it (provided you haven’t blocked it). But in RSS, it is you who has the control. If you decide to stop visiting a website, then all you need to do is just delete the link from your app. Emails require you to unsubscribe, where the company can collect your feedback and so.

Disadvantages

User is not in control. To let go off the updates, you need to unsubscribe from the company. It can be a big hassle.
If the email addresses stored in the company database is breached, then it will be a threat to your privacy and security.
Requires an email server. It can be costly, so not affordable by all.

So RSS

The advantages of RSS are the disadvantages of the email. Full control for the user and no need of any email server. For an individual developer who is not any interested in user’s usage (so called tracking), RSS is the best.

Many RSS aggregators exist, including free and open source ones. If you are on Android, you can check out Feeder. The steps to add a website to an app, depends on the app in particular.

Conclusion

The article was meant to brief introduction to RSS. You can learn more about it from the world wild web. RSS is an old technology, it was once at the peak of usage. With the growth in personalized and targeted advertisements, it has gone to a decline. These days, especially among the personal blogs and websites, there is a growing interest in RSS because of privacy and control of user. Hope it continues and enriches.

Resume

Arun Mani J (j.arunmani@proton.me) — Mon, 01 Jan 0001 00:00:00 +0000

Arun Mani J

The Palace of Illusions

Plato and a Platypus Walk Into a Bar: Understanding Philosophy Through Jokes

Twenty Two Goblins

The Midnight Library

State of Stateless - DebConf 2023

Creating Blurred Background Image using ImageMagick

Introduction

The Need

Just Crop?

Blurred Background Image

Why Blur?

Automating It Using ImageMagick

The Command

Image Sequence and Parenthesis

Blurring

Ratio-Preserving Resize

Delete Input

Gravity

Compose

Conclusion

Jaya

The Graveyard Book

Publishing Book with Emacs and OrgMode

Brief About The Book (Can be skipped)

It Started In My First Year

How To Write A Book?

Here Comes Emacs

So Org mode and LaTeX

Shaping for Lab Manual

Maintenance of Programs

Creation of Flowcharts

Getting a Publisher

Sudden Fear

Merging Chapters

Final Steps

The Right Choice

Technical Details

Setting up Emacs

Setting up LaTeX

Miscellaneous Packages

Enabling Syntax Highlighting

Organization

The Template

Creating Table of Contents

Designing Front Matter

Finishing Up

Conclusion

Footnotes

The Nightingale

Looking for Alaska

Malgudi Days

The Shadow of the Wind

Norse Mythology

Let's Make A Silly JSON-like Parser

Understanding Our Data

Jason - The Parser

The Preparation

Defining Constants

Our Functions

Extracting Value

Skipping Space

Extracting Number

Extracting String

Extracting List

Extracting JSON

Extracting null

Are We Done Yet?

Are We Done Yet??!!

Conclusion

Aayra

aurthy

Aashii

DoNotRepeatBot

Debian Day’22, Bishop Heber College, Trichy

One Hundred Years of Solitude

The Pregnant King

Educated

Norwegian Wood

Quack

Extracting `null`