Blog

If you’re running Elasticsearch in Docker locally, you might encounter a frustrating issue where your cluster suddenly goes into a red (unhealthy) state, even though everything seems fine. This often happens when your local disk usage exceeds 90%. I tend to mount the elasticsearch data directory into my host file system to preserve data. version: "3.9" services: elasticsearch: image: elasticsearch:9.0.2 environment: - discovery.type=single-node - ES_JAVA_OPTS=-Xms1g -Xmx1g - xpack.security.enabled=false volumes: - ./es_data:/usr/share/elasticsearch/data ports: - target: 9200...

Certificate pinning code for Java package org.example; import javax.net.ssl.*; import java.io.*; import java.net.URL; import java.security.MessageDigest; import java.security.cert.Certificate; import java.security.cert.X509Certificate; import java.util.Base64; public class CertificatePinningClient { // SHA-256 fingerprint of the expected certificate // You need to replace this with the actual certificate fingerprint of madhur.co.in private static final String EXPECTED_CERT_SHA256 = "YOUR_CERT_FINGERPRINT_HERE"; public static void main(String[] args) { try { String fingerprint = getCertificateFingerprint("https://localhost:8443"); System.out.println(fingerprint); // String response = makeSecureRequest("https://madhur.co.in"); // System.out.println("Response: " + response);...

I wanted to combine videos using the tool. I had two videos in 9x16 format and wanted to combine side by side. With a little bit of trial and error, I was able to achieve this using ffmpeg The exact command is ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[0:v]scale=540:960:force_original_aspect_ratio=decrease,pad=540:960:(ow-iw)/2:(oh-ih)/2[v0];[1:v]scale=540:960:force_original_aspect_ratio=decrease,pad=540:960:(ow-iw)/2:(oh-ih)/2[v1];[v0][v1]hstack=inputs=2[v]" -map "[v]" -map 0:a -c:v libx264 -c:a aac output.mp4 Breaking Down the Command Let’s understand what each part of this complex command does: Input Files -i...

Recently we faced an interesting issue in our data pipeline where we observed Kafka consumer lag suddenly dropping whenever it reached a certain threshold. This behavior was unexpected and potentially concerning, as consumer lag—the difference between the latest message produced and the message currently being processed by consumers—should typically decrease gradually as consumers process messages. The Problem Our application was relatively simple, with two instances of pods running in Kubernetes. Since we didn’t need high...

Recently we faced an issue, where there were some hidden characters inside the hostname of a configuration store of our backend services. Because of this, we were getting continously getting DNS resolution errors. One quick way to reveal such hidden characters is to run the command echo "string" | cat -v For example try copying the belwo string into dig or a whois website. madhur.​co.​in The above string contains hidden characters which can be reveled...

Quick update on my personal homelab server. I set up a dashboard using homepage to monitor and manage all my services in one place. Though it was a bit tedious to set up, it turned out well. I also setup a Debian VM on my proxmox to host some of the applications on docker. The applications I am hosting through docker are: FreshRSS - RSS Aggregator Memos - Self-hosted note-taking application for capturing and organizing...

I was playing around with Qtile when it suddenly stopped working. I tried everything to troubleshoot but wasn’t getting any clues. There was nothing in ~/.local/share/qtile/qtile.log I found this entry in systemd journal through journalctl | grep qtile Here’s what the logs showed: Dec 24 09:14:45 madhur-b550mds3h systemd-coredump[26096]: Process 26057 (qtile) of user 1000 terminated abnormally with signal 11/SEGV, processing... Dec 24 09:14:45 madhur-b550mds3h systemd-coredump[26097]: Process 26057 (qtile) of user 1000 dumped core. Dec 24...

I needed to resize a disk in my Proxmox VM. Specifically, I was working with a Debian 12 (Bookworm) VM created through Proxmox VE Helper Scripts. These VMs come with a default size of 2 GB, even when using the advanced installation option. Unfortunately, there’s no way to select the disk size during the initial installation process. Pre-Requisites First, install the required tools: apt install parted fdisk Steps to Resize the disk Resize the disk...

If you are looking to copy huge amount of data to/from AWS S3 Look no further than s5cmd Download the entire s3 bucket to local s5cmd cp s3://bucket/ . I was able to download almost 10 GB folder in couple of minutes, which is around 83MBps ~ 10 MB/s Whereas aws s3 cp --recursive s3://bucket/ . was giving me around 500 KB/s

Spring data MongoDB adds a _class attribute to every document written to MongoDB If we want to prvent adding Spring Data MongoDB from adding _class attribute, we can configure this behaviour through: @Configuration public class MongoConfig extends AbstractMongoClientConfiguration { @Override protected boolean autoIndexCreation() { return true; } @Bean public MappingMongoConverter mappingMongoConverter(MongoDatabaseFactory databaseFactory, MongoMappingContext mappingContext) { DbRefResolver dbRefResolver = new DefaultDbRefResolver(databaseFactory); MappingMongoConverter converter = new MappingMongoConverter(dbRefResolver, mappingContext); converter.setTypeMapper(new DefaultMongoTypeMapper(null)); return converter; } }