[{"content":"Credentials 🔗 Certificate 🔗 Credly Badge 🎬 YouTube Video Introduction I\u0026rsquo;ve passed the AWS Solutions Architect - Associate certification exam with just a month of preparation while pursuing MSc in Applied Computing at the University of Toronto and working as a DevOps Intern at 16Bit, a med-tech startup based in Toronto. In this article, I’ll be sharing everything about this exam, my preparation strategies and tips. So, if you have plans on taking this exam anytime soon, read this article till the end.\nWhat is AWS Developer - Associate Certification With the rise of Cloud Computing, companies have constantly been shifting from running their infrastructure on-premise, which is hard to scale and manage, to running them on cloud, which offers far more elasticity in terms of scaling and resiliency in case of disasters.\nThe AWS Developer - Associate certification validates your expertise in developing and maintaining applications on AWS, which is the leading cloud provider today. It focuses on AWS core services, development tools and deployment practices. It can help developers enhance their skills, increase their earning potential, and demonstrate their proficiency to potential employers.\nExam Format In this exam, you will get 65 questions and 130 minutes to answer them which means you get 2 mins per question. Out of the total questions, 15 will be experimental (ungraded). But you will not know which ones are experimental, otherwise you will skip them. So, you need to attempt every question on the test.\nYou will be graded on a percentile basis on a scale of 100 to 1000 where you will nead to score more than 720 to pass the exam. Based on this information, you can approximate that you will have to answer about 72% of the questions correctly in order to pass. You can refer this link for more details on how scoring works.\nThis exam has a pass / fail criteria. If you pass the exam, your score doesn\u0026rsquo;t really matter. It will only be written on your score report for your reference. It will not be mentioned anywhere on the certificate.\nThe cost of taking this exam is 150 USD but I got 50% off, so I paid just 75 USD. The way it works is that if you passed your first AWS certification exam, then you can get 50% off on the second one if you take it within a year of taking the first certification. Since, I passed the AWS Solution’s Architect - Associate Certification exam in the month of May last year, I was eligible for the discount.\nMy Preparation Strategy I prepared for about 4-6 hours everyday for a months while being a grad student and working as a DevOps intern. I took the AWS Developer - Associate Course by Stephane Maarek which is available on Udemy. Preparing for the DVA exam was a bit easy for me as I had already passed the SAA exam, because the concepts overlap to some degree between the two certification exams.\nOnce I was done with the course, I bought three sets of practice tests on Udemy that are provided by:\nStephane Maarek Jon Bonso Neal Davis Each of these practice test packages contain 6 practice tests. Additionally, a free practice test is provided with the course.\nWhile going through the course, I took easy to follow notes in Notion. Additionally, while taking the practice tests, I saved the questions along with their explanations, into a Notion page, for the questions that I got wrong and the questions that I found difficult to answer. This would come in handy later when I revise everything before the exam.\nI also created a notion page for quick notes to include information that’s hard to remember, things like message retention duration of a Kinesis Data Stream or the IOPS limits for the different EBS volume types. I just glanced through this stuff before taking the actual test to have all that information fresh in my memory.\n🔗 My notes\nTaking the Test You can take the test either at a testing center or online at the comfort of your home. I would suggest you go for a testing center if it is available nearby. The experience is so smooth and you don\u0026rsquo;t have to worry about internet connectivity or proctoring issues. I took the test at a testing center and my experience was good.\nLast year, I took the AWS SAA exam online from my home and my experience was pretty bad. The proctor asked me not to move my lips while reading the questions. So, the proctoring was way too strict. I was stressed and mindful about my body position, and was not able to focus well on the actual test. Another problem with taking the test online is that if the proctor cancels your exam, you won\u0026rsquo;t get a refund. You will have to rebook and retake the test at a later date.\nTips for taking the AWS SAA exam Take as many practice tests as you can. They will give you an idea of the kind of topics that come up in the exam most often. Also, the questions in these practice tests match very well with the ones appearing on the actual exam.\nThe amount of information that you will have to go through to prepare for this exam is enormous. You not only need a good understanding of the various AWS resources and architectures, but you will also have to remember a lot of information. So, keep at least 3 days to revise your notes before the exam.\nThat\u0026rsquo;s all folks That was all about the AWS Developer - Associate Certification exam. Thanks a lot for reading!\n","permalink":"https://fred.devs.surf/blog/aws-dva-certification/","summary":"Credentials 🔗 Certificate 🔗 Credly Badge 🎬 YouTube Video Introduction I\u0026rsquo;ve passed the AWS Solutions Architect - Associate certification exam with just a month of preparation while pursuing MSc in Applied Computing at the University of Toronto and working as a DevOps Intern at 16Bit, a med-tech startup based in Toronto. In this article, I’ll be sharing everything about this exam, my preparation strategies and tips. So, if you have plans on taking this exam anytime soon, read this article till the end.","title":"I passed the AWS DVA Certification Exam"},{"content":"Credentials 🔗 Certificate 🔗 Credly Badge 🎬 YouTube Video Introduction I\u0026rsquo;ve passed the AWS Solutions Architect - Associate certification exam with just 2 months of preparation while working full-time as a software engineer. In this article, I’ll be sharing everything about this exam, my preparation strategies and tips. So, if you have plans on taking this exam anytime soon, read this article till the end.\nWhat is AWS Solutions Architect Associate Certification With the rise of Cloud Computing, companies have constantly been shifting from running their infrastructure on-premise, to running them on cloud, which offers far more elasticity in terms of scaling and resiliency in case a disaster strikes.\nThe AWS Solutions Architect - Associate certification validates your ability to design and deploy well-architected solutions on AWS, which is the leading cloud provider today. In simple terms, this exam tests your ability to propose an architecture given a specific scenario. For example: a company wants their application to continue running even if an entire AWS region, where that application was hosted, is down due to a disaster. So, how would you design their infrastructure around this use case?\nExam Format In this exam, you will get 65 questions and 130 minutes to answer them which means you get 2 mins per question. You will be graded on a percentile basis on a scale of 100 to 1000 where you will nead to score more than 720 to pass the exam. Based on this information, you can approximate that you will have to answer about 72% of the questions correctly in order to pass. You can refer this link for more details on how scoring works.\nThis exam has a pass / fail criteria. If you pass the exam, your score doesn\u0026rsquo;t really matter. It will only be written on your score report for your reference. It will not be mentioned anywhere on the certificate.\nThe cost of taking this exam is 150 USD and with taxes it comes up to about 177 USD. Since I took the exam in India, the amount for me was around 13,500 INR.\nMy Preparation Strategy I prepared for about 2 months while having a full-time job and simultaneously running my 🎬 YouTube channel where I post resourceful videos every week. During these 2 months of preparatory period, I studied for about 2 hours on weekdays and 4-6 hours on weekends.\nFor preparation, I took the AWS Solutions Architect Associate Course by Stephane Maarek which is available on Udemy. While taking this course, I dumped all of the information available in the course into a Notion page.\nOnce I was done with the course, I bought three practice test packages for the AWS SAA exam on Udemy that are provided by:\nStephane Maarek Jon Bonso Neal Davis Each of these practice test packages contain 6 practice tests. Additionally, a free practice test is provided with the course.\nWhile taking the practice tests, I dumped the questions along with their explanations, into a Notion page, for the questions that I got wrong and the questions that I found difficult to answer. This would come in handy later when I revise everything before the exam.\nUntil now, everything was entangled in my head as I had not consumed information in an organized manner. So, I consolidated all of the information from the course and the practice tests into dense concise notes that, instead of Notion, I took on another note-taking app called Obsidian. I’ll explain why in another video. For the sake of the AWS exam, you can take your notes anywhere.\nIf you want my notes, you will have to wait for some time until I figure out a way to share my Obsidian notes in a presentable format. They cannot be directly shared like Notion pages.\nConsolidating my notes took about a week and while doing so I went through all of the information again but this time with a much more idea of the concepts. Everything started making sense and I felt confident to take the AWS SAA exam. So, I revised my consolidated notes once and took the exam the next day.\nTaking the Test You can either take the test offline at a testing center or online at the comfort of your home. I would suggest you take the test offline if you have testing centers in your area. If not, then you can take the online route. I had to take this test online as there are no testing centers nearby.\nDo keep in mind that the proctoring in the online test is extremely strict and if the proctor cancels your exam, you won\u0026rsquo;t get a refund. In such a scenario, you will have to rebook and retake the test at a later date.\nTips for taking the AWS SAA exam Take as many practice tests as you can. They will give you an idea of the kind of topics that come up in the exam most often. Also, the questions in these practice tests match very well with the ones appearing on the actual exam.\nThe amount of information that you will have to go through to prepare for this exam is enormous. You not only need a good understanding of the various AWS resources and architectures, but you will also have to remember a lot of information. So, filtering out the irrelevant details from the dumped information and making concise notes, that you can easily revise within 1 or 2 days is crucial for this exam.\nThat\u0026rsquo;s all folks That was all about the AWS Solutions Architect - Associate exam. Up next, I have plans to take the AWS Developer Associate exam which focuses on the development aroud AWS services. As a personal milestone, I want to clear the AWS Developer Associate exam before I move to Canada 🇨🇦 for my MS.\n","permalink":"https://fred.devs.surf/blog/aws-saa-certification/","summary":"Credentials 🔗 Certificate 🔗 Credly Badge 🎬 YouTube Video Introduction I\u0026rsquo;ve passed the AWS Solutions Architect - Associate certification exam with just 2 months of preparation while working full-time as a software engineer. In this article, I’ll be sharing everything about this exam, my preparation strategies and tips. So, if you have plans on taking this exam anytime soon, read this article till the end.\nWhat is AWS Solutions Architect Associate Certification With the rise of Cloud Computing, companies have constantly been shifting from running their infrastructure on-premise, to running them on cloud, which offers far more elasticity in terms of scaling and resiliency in case a disaster strikes.","title":"I passed the AWS SAA Certification Exam"},{"content":"✏️ Intro If you’re like me who loves reading books on Kindle, you might have wondered how you could extract your highlights in an organized way and save them as notes. At least I did. You see, I use Notion as my primary note-taking/productivity management app and I already have a database of all the books that I have read so far and also the ones that I am planning to read next.\nAnd since each of these book entries in the Notion database is a page in itself, I thought why not populate them with the highlights that I made in Kindle while reading them. The only problem was Kindle stores all of the highlights in a text file (My Clippings.txt) which as you can see contains a tonne of useless information like the book location, where the highlight was made, and when it was made.\nI needed to find a way to filter out the highlights, group them by the book title and send them to my Notion book database. Not only that, all of this should happen automatically with minimal human effort. So, over the past two weekends, I spent the majority of my time coding and I’m finally ready with an app that would allow readers to seamlessly transfer all of their highlights to Notion. Let’s take a look\u0026hellip;\n🤖 Node Environment You need a stable version of Node JS, installed locally, to run this app. I have tested this on Node versions 16 and 14, and it has worked flawlessly on both of them. So, before proceeding to the next steps, make sure you have a stable version of Node installed. I’m not going to explain the environment setup in this article because the installation process might differ for different operating systems. You can easily learn that on Google.\n⚙️ Setup Follow the steps given below to set up the Kindle to Notion app on your local system.\nCopy my Books Database Template to your Notion dashboard. The app requires some fields (Title, Author, and Book Name) to be present in the database in order for the highlight sync to work properly. So, you can either create your own database having these fields or you can just copy mine using the template I provided above.\nClone the GitHub Repository to your local system and install the dependencies.\ngit clone https://github.com/arkalim/kindle-to-notion.git cd kindle-to-notion npm install Rename these files or folders by removing .example extensions as shown below. The original files/folders in my local repo contained data that was either sensitive or specific to my highlights. So, I created empty aliases of them with .example extensions and committed them to GitHub.\n‣ cache.example ➡ cache\n‣ data.example ➡ data\n‣ .env.example ➡ .env\nGet your Notion API key at the Notion Integrations page and create a new internal integration. Integrations allow us to access a portion of our Notion workspace using a secret token called the Notion API key (Internal Integration Token).\nGo to your Notion dashboard. Navigate to the Books database. Click on Share in the top right-hand corner and invite the integration you just created. This will allow the integration to edit the Books database using the Notion API key that we got in the previous step. Copy the link to the Notion Books database and extract the Database Id as shown below. The database id is nothing but all of the gibberish between the last / and the ?. This is required by the app to perform CRUD operations on this database. Original Link: https://www.notion.so/arkalim/346be84507ff482b80fceb4024deadc2?v=e868075eaf5749bc941e617e651295fb Database Id: 346be84507ff482b80fceb4024deadc2 So, now you have the Notion API key as well as the Database Id. Now, populate these variables in the .env file. Storing this sensitive information in .env ensures that it won’t get exposed to the rest of the world if you commit your local repo to GitHub as .gitignore has been configured to ignore .env during commits. NOTION_API_KEY=your-notion-api-key BOOK_DB_ID=your-book-database-id Connect your Kindle to your computer. Navigate to Kindle ➡ documents and copy My Clippings.txt. Replace my My Clippings.txt in resources folder with yours. 🔁 Sync Highlights Finally, we are at the end of the setup section. You are now ready to sync your Kindle highlights to Notion. Open a terminal in your local repository and run the following command to watch your highlights teleport!\nnpm start ❗️For Nerds Every highlight made on Kindle is appended at the end of My Clippings.txt. RegEx has been used extensively throughout the application to parse and filter this text file. cache is a folder that contains the local cache to prevent the app from resyncing old highlights. data is a folder that contains the API response logs. .env is a file containing the environment variables like the Notion API key and the Database Id. Book Name is used as the primary key to facilitate upsert operation in the Notion database. Book Name corresponds to the title of the book in My Clippings.txt. So, this field should be left untouched. However, the other fields like Title, Author, Date Started, Date Finished, Status, and Genre could be modified as per your wish. The app maintains a local cache in the file sync.json present in the cache folder. This JSON file is updated at the end of each sync. This is done to prevent the app from resyncing the old highlights. If no new highlights have been made, no sync takes place. In case you wish to sync every book all over again, you need to empty the array present in sync.json and delete all the highlights present in your Notion database before running the sync. Responses from Notion API calls are exported to files with .json extensions in data folder. This was done to mitigate the problem of effectively logging JSON objects in the console (terminal). That’s all folks! If you made it till here, hats off to you! In this article, we learned how to set up Kindle to Notion app on our local system and use it to sync our Kindle highlights to the Notion Books database. If you want me to write more detailed articles explaining the inner workings of this app, drop a comment below. I write articles regularly so you should consider following me to get more such articles in your feed. Thanks a lot for reading!\n","permalink":"https://fred.devs.surf/blog/kindle-to-notion/","summary":"✏️ Intro If you’re like me who loves reading books on Kindle, you might have wondered how you could extract your highlights in an organized way and save them as notes. At least I did. You see, I use Notion as my primary note-taking/productivity management app and I already have a database of all the books that I have read so far and also the ones that I am planning to read next.","title":"Kindle to Notion"},{"content":"Introduction Ever wondered how Instagram applies stunning filters to your face? The software detects key points on your face and projects a mask on top. This tutorial will guide you on how to build one such software using Pytorch.\nDataset In this tutorial, we will use the official DLib Dataset which contains 6666 images of varying dimensions. Additionally, labels_ibug_300W_train.xml (comes with the dataset) contains the coordinates of 68 landmarks for each face. The script below will download the dataset and unzip it in Colab Notebook.\nif not os.path.exists(\u0026#39;/content/ibug_300W_large_face_landmark_dataset\u0026#39;): !wget http://dlib.net/files/data/ibug_300W_large_face_landmark_dataset.tar.gz !tar -xvzf \u0026#39;ibug_300W_large_face_landmark_dataset.tar.gz\u0026#39; !rm -r \u0026#39;ibug_300W_large_face_landmark_dataset.tar.gz\u0026#39; Here is a sample image from the dataset. We can see that the face occupies a very small fraction of the entire image. If we feed the full image to the neural network, it will also process the background (irrelevant information), making it difficult for the model to learn. Therefore, we need to crop the image and feed only the face portion.\nData Preprocessing To prevent the neural network from overfitting the training dataset, we need to randomly transform the dataset. We will apply the following operations to the training and validation dataset:\nSince the face occupies a very small portion of the entire image, crop the image and use only the face for training. Resize the cropped face into a (224x224) image. Randomly change the brightness and saturation of the resized face. Randomly rotate the face after the above three transformations. Convert the image and landmarks into torch tensors and normalize them between [-1, 1]. class Transforms(): def __init__(self): pass def rotate(self, image, landmarks, angle): angle = random.uniform(-angle, +angle) transformation_matrix = torch.tensor([ [+cos(radians(angle)), -sin(radians(angle))], [+sin(radians(angle)), +cos(radians(angle))] ]) image = imutils.rotate(np.array(image), angle) landmarks = landmarks - 0.5 new_landmarks = np.matmul(landmarks, transformation_matrix) new_landmarks = new_landmarks + 0.5 return Image.fromarray(image), new_landmarks def resize(self, image, landmarks, img_size): image = TF.resize(image, img_size) return image, landmarks def color_jitter(self, image, landmarks): color_jitter = transforms.ColorJitter(brightness=0.3, contrast=0.3, saturation=0.3, hue=0.1) image = color_jitter(image) return image, landmarks def crop_face(self, image, landmarks, crops): left = int(crops[\u0026#39;left\u0026#39;]) top = int(crops[\u0026#39;top\u0026#39;]) width = int(crops[\u0026#39;width\u0026#39;]) height = int(crops[\u0026#39;height\u0026#39;]) image = TF.crop(image, top, left, height, width) img_shape = np.array(image).shape landmarks = torch.tensor(landmarks) - torch.tensor([[left, top]]) landmarks = landmarks / torch.tensor([img_shape[1], img_shape[0]]) return image, landmarks def __call__(self, image, landmarks, crops): image = Image.fromarray(image) image, landmarks = self.crop_face(image, landmarks, crops) image, landmarks = self.resize(image, landmarks, (224, 224)) image, landmarks = self.color_jitter(image, landmarks) image, landmarks = self.rotate(image, landmarks, angle=10) image = TF.to_tensor(image) image = TF.normalize(image, [0.5], [0.5]) return image, landmarks Dataset Class Now that we have our transformations ready, let’s write our dataset class. The labels_ibug_300W_train.xml contains the image path, landmarks and coordinates for the bounding box (for cropping the face). We will store these values in lists to access them easily during training. In this tutorial, the neural network will be trained on grayscale images.\nclass FaceLandmarksDataset(Dataset): def __init__(self, transform=None): tree = ET.parse(\u0026#39;ibug_300W_large_face_landmark_dataset/labels_ibug_300W_train.xml\u0026#39;) root = tree.getroot() self.image_filenames = [] self.landmarks = [] self.crops = [] self.transform = transform self.root_dir = \u0026#39;ibug_300W_large_face_landmark_dataset\u0026#39; for filename in root[2]: self.image_filenames.append(os.path.join(self.root_dir, filename.attrib[\u0026#39;file\u0026#39;])) self.crops.append(filename[0].attrib) landmark = [] for num in range(68): x_coordinate = int(filename[0][num].attrib[\u0026#39;x\u0026#39;]) y_coordinate = int(filename[0][num].attrib[\u0026#39;y\u0026#39;]) landmark.append([x_coordinate, y_coordinate]) self.landmarks.append(landmark) self.landmarks = np.array(self.landmarks).astype(\u0026#39;float32\u0026#39;) assert len(self.image_filenames) == len(self.landmarks) def __len__(self): return len(self.image_filenames) def __getitem__(self, index): image = cv2.imread(self.image_filenames[index], 0) landmarks = self.landmarks[index] if self.transform: image, landmarks = self.transform(image, landmarks, self.crops[index]) landmarks = landmarks - 0.5 return image, landmarks dataset = FaceLandmarksDataset(Transforms()) Note: landmarks = landmarks - 0.5 is done to zero-centre the landmarks as zero-centred outputs are easier for the neural network to learn.\nThe output of the dataset after preprocessing will look something like this (landmarks have been plotted on the image).\nNeural Network We will use the ResNet18 as the basic framework. We need to modify the first and last layers to suit our purpose. In the first layer, we will make the input channel count as 1 for the neural network to accept grayscale images. Similarly, in the final layer, the output channel count should equal 68 * 2 = 136 for the model to predict the (x, y) coordinates of the 68 landmarks for each face.\nclass Network(nn.Module): def __init__(self,num_classes=136): super().__init__() self.model_name=\u0026#39;resnet18\u0026#39; self.model=models.resnet18() self.model.conv1=nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3, bias=False) self.model.fc=nn.Linear(self.model.fc.in_features, num_classes) def forward(self, x): x=self.model(x) return x Training the Neural Network We will use the Mean Squared Error between the predicted landmarks and the true landmarks as the loss function. Keep in mind that the learning rate should be kept low to avoid exploding gradients. The network weights will be saved whenever the validation loss reaches a new minimum value. Train for at least 20 epochs to get the best performance.\nnetwork = Network() network.cuda() criterion = nn.MSELoss() optimizer = optim.Adam(network.parameters(), lr=0.0001) loss_min = np.inf num_epochs = 10 start_time = time.time() for epoch in range(1,num_epochs+1): loss_train = 0 loss_valid = 0 running_loss = 0 network.train() for step in range(1,len(train_loader)+1): images, landmarks = next(iter(train_loader)) images = images.cuda() landmarks = landmarks.view(landmarks.size(0),-1).cuda() predictions = network(images) # clear all the gradients before calculating them optimizer.zero_grad() # find the loss for the current step loss_train_step = criterion(predictions, landmarks) # calculate the gradients loss_train_step.backward() # update the parameters optimizer.step() loss_train += loss_train_step.item() running_loss = loss_train/step print_overwrite(step, len(train_loader), running_loss, \u0026#39;train\u0026#39;) network.eval() with torch.no_grad(): for step in range(1,len(valid_loader)+1): images, landmarks = next(iter(valid_loader)) images = images.cuda() landmarks = landmarks.view(landmarks.size(0),-1).cuda() predictions = network(images) # find the loss for the current step loss_valid_step = criterion(predictions, landmarks) loss_valid += loss_valid_step.item() running_loss = loss_valid/step print_overwrite(step, len(valid_loader), running_loss, \u0026#39;valid\u0026#39;) loss_train /= len(train_loader) loss_valid /= len(valid_loader) print(\u0026#39;\\n--------------------------------------------------\u0026#39;) print(\u0026#39;Epoch: {} Train Loss: {:.4f} Valid Loss: {:.4f}\u0026#39;.format(epoch, loss_train, loss_valid)) print(\u0026#39;--------------------------------------------------\u0026#39;) if loss_valid \u0026lt; loss_min: loss_min = loss_valid torch.save(network.state_dict(), \u0026#39;/content/face_landmarks.pth\u0026#39;) print(\u0026#34;\\nMinimum Validation Loss of {:.4f} at epoch {}/{}\u0026#34;.format(loss_min, epoch, num_epochs)) print(\u0026#39;Model Saved\\n\u0026#39;) print(\u0026#39;Training Complete\u0026#39;) print(\u0026#34;Total Elapsed Time : {} s\u0026#34;.format(time.time()-start_time)) Predict on Unseen Data Use the code snippet below to predict landmarks in unseen images.\nimport time import cv2 import os import numpy as np import matplotlib.pyplot as plt from PIL import Image import imutils import torch import torch.nn as nn from torchvision import models import torchvision.transforms.functional as TF ####################################################################### image_path = \u0026#39;pic.jpg\u0026#39; weights_path = \u0026#39;face_landmarks.pth\u0026#39; frontal_face_cascade_path = \u0026#39;haarcascade_frontalface_default.xml\u0026#39; ####################################################################### class Network(nn.Module): def __init__(self,num_classes=136): super().__init__() self.model_name=\u0026#39;resnet18\u0026#39; self.model=models.resnet18(pretrained=False) self.model.conv1=nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3, bias=False) self.model.fc=nn.Linear(self.model.fc.in_features,num_classes) def forward(self, x): x=self.model(x) return x ####################################################################### face_cascade = cv2.CascadeClassifier(frontal_face_cascade_path) best_network = Network() best_network.load_state_dict(torch.load(weights_path, map_location=torch.device(\u0026#39;cpu\u0026#39;))) best_network.eval() image = cv2.imread(image_path) grayscale_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) display_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) height, width,_ = image.shape faces = face_cascade.detectMultiScale(grayscale_image, 1.1, 4) all_landmarks = [] for (x, y, w, h) in faces: image = grayscale_image[y:y+h, x:x+w] image = TF.resize(Image.fromarray(image), size=(224, 224)) image = TF.to_tensor(image) image = TF.normalize(image, [0.5], [0.5]) with torch.no_grad(): landmarks = best_network(image.unsqueeze(0)) landmarks = (landmarks.view(68,2).detach().numpy() + 0.5) * np.array([[w, h]]) + np.array([[x, y]]) all_landmarks.append(landmarks) plt.figure() plt.imshow(display_image) for landmarks in all_landmarks: plt.scatter(landmarks[:,0], landmarks[:,1], c = \u0026#39;c\u0026#39;, s = 5) plt.show() ⚠️ The above code snippet will not work in Colab Notebook as some functionality of the OpenCV is not supported in Colab yet. To run the above cell, use your local machine.\nOpenCV Harr Cascade Classifier is used to detect faces in an image. Object detection using Haar Cascades is a machine learning-based approach where a cascade function is trained with a set of input data. OpenCV already contains many pre-trained classifiers for face, eyes, pedestrians, and many more. In our case, we will be using the face classifier for which you need to download the pre-trained classifier XML file and save it to your working directory.\nDetected faces in the input image are then cropped, resized to (224, 224) and fed to our trained neural network to predict landmarks in them.\nThe predicted landmarks in the cropped faces are then overlayed on top of the original image. The result is the image shown below. Pretty impressive, right!\nSimilarly, landmarks detection on multiple faces:\nHere, you can see that the OpenCV Harr Cascade Classifier has detected multiple faces including a false positive (a fist is predicted as a face). So, the network has plotted some landmarks on that.\nThat’s all folks! If you made it till here, hats off to you! You just trained your very own neural network to detect face landmarks in any image. Try predicting face landmarks on your webcam feed!!\nColab Notebook The complete code can be found in the interactive Colab Notebook.\n","permalink":"https://fred.devs.surf/blog/face-landmarks-detection/","summary":"Introduction Ever wondered how Instagram applies stunning filters to your face? The software detects key points on your face and projects a mask on top. This tutorial will guide you on how to build one such software using Pytorch.\nDataset In this tutorial, we will use the official DLib Dataset which contains 6666 images of varying dimensions. Additionally, labels_ibug_300W_train.xml (comes with the dataset) contains the coordinates of 68 landmarks for each face.","title":"Face Landmarks Detection using CNN"},{"content":"Introduction to machine learning In the traditional hard-coded approach, we program a computer to perform a certain task. We tell it exactly what to do when it receives a certain input. In mathematical terms, this is like saying that we write the f(x) such that when users feed the input x into f(x), it gives the correct output y.\nIn machine learning, however, we have a large set of inputs x and corresponding outputs y but not the function f(x). The goal here is to find the f(x) that transforms the input x into the output y. Well, that’s not an easy job. In this article, we will learn how this happens.\nDataset To visualize the dataset, let’s make our synthetic dataset where each data point (input x) is 3 dimensional, making it suitable to be plotted on a 3D chart. We will generate 250 points (cluster 0) in a cluster centered at the origin (0, 0, 0). A similar cluster of 250 points (cluster 1) is generated but not centered at the origin. Both clusters are relatively close but there is a clear separation as seen in the image below. These two clusters are the two classes of data points. The big green dot represents the centroid of the whole dataset.\nAfter generating the dataset, we will normalize it by subtracting the mean and dividing by the standard deviation. This is done to zero-center the data and map values in each dimension in the dataset to a common scale. This speeds up the learning.\nThe data will be saved in an array X containing the 3D coordinates of normalized points. We will also generate an array Y with the value either 0 or 1 at each index depending on which cluster the 3D point belongs.\nLearnable Function Now that we have our data ready, we can say that we have the x and y. We know that the dataset is linearly separable implying that there is a plane that can divide the dataset into the two clusters, but we don’t know what the equation of such an optimal plane is. For now, let’s just take a random plane.\nThe function f(x) should take a 3D coordinate as input and output a number between 0 and 1. If this number is less than 0.5, this point belongs to cluster 0 otherwise, it belongs to cluster 1. Let’s define a simple function for this task.\nx: input tensor of shape (num_points, 3)W: Weight (parameter) of shape (3, 1) chosen randomlyB: Bias (parameter) of shape (1, 1) chosen randomlySigmoid: A function that maps values between 0 and 1\nLet’s take a moment to understand what this function means. Before applying the sigmoid function, we are simply creating a linear mapping from the 3D coordinate (input) to 1D output. Therefore, this function will squish the whole 3D space onto a line meaning that each point in the original 3D space will now be lying somewhere on this line. Since this line will extend to infinity, we map it to [0, 1] using the Sigmoid function. As a result, for each given input, f(x) will output a value between 0 and 1.\nRemember that W and B are chosen randomly and so the 3D space will be squished onto a random line. The decision boundary for this transformation is the set of points that make f(x) = 0.5. Think why! As the 3D space is being squished onto a 1D line, a whole plane is mapped to the value 0.5 on the line. This plane is the decision boundary for f(x). Ideally, it should divide the dataset into two clusters but since W and B are randomly chosen, this plane is randomly oriented as shown below.\nOur goal is to find the right values for W and B that orients this plane (decision boundary) in such a way that it divides the dataset into the two clusters. This when done, yields a plane as shown below.\nLoss So, we are now at the starting point (random decision boundary) and we have defined the goal. We need a metric to decide how far we are from the goal. The output of the classifier is a tensor of shape (num_points, 1) where each value is between [0, 1]. If you think carefully, these values are just the probabilities of the points belonging to cluster 1. So, we can say that:\nf(x) = P(x belongs to cluster 1) 1-f(x) = P(x belongs to cluster 0) It wouldn’t be wrong to say that [1-f(x), f(x)] forms a probability distribution over the clusters 0 and cluster 1 respectively. This is the predicted probability distribution. We know for sure which cluster every point in the dataset belongs to (from y). So, we also have the true probability distribution as:\n[0, 1] when x belongs to the cluster 1 [1, 0] when x belongs to the cluster 0 A good metric to calculate the incongruity between two probability distributions is the Cross-Entropy function. As we are dealing with just 2 classes, we can use Binary Cross-Entropy (BCE). This function is available in PyTorch’s torch.nn module. If the predicted probability distribution is very similar to the true probability distribution, this function returns a small value and vice versa. We can average this value for all the data points and use it as a parameter to test how the classifier is performing.\nThis value is called the loss and mathematically, our goal now is to minimize this loss.\nTraining Now that we have defined our goal mathematically, how do we reach our goal practically? In other words, how do we find optimal values for W and B? To understand this, we will take a look at some basic calculus. Recall that we currently have random values for W and B. The process of learning or training or reaching the goal or minimizing the loss can be divided into two steps:\nForward-propagation: We feed the dataset through the classifier f(x) and use BCE to find the loss. Backpropagation: Using the loss, adjust the values of W and B to minimize the loss. The above two steps will be repeated over and over again until the loss stops decreasing. In this condition, we say that we have reached the goal!\nBackpropagation Forward propagation is simple and already discussed above. However, it is essential to take a moment to understand backpropagation as it is the key to machine learning. Recall that we have 3 parameters (variables) in W and 1 in B. So, in total, we have 4 values to optimize.\nOnce we have the loss from forward-propagation, we will calculate the gradients of the loss function with respect to each variable in the classifier. If we plot the loss for different values of each parameter, we can see that the loss is minimum at a particular value for each parameter. I have plotted the loss vs parameter for each parameter.\nAn important observation to make here is that the loss is minimized at a particular value for each of these parameters as shown by the red dot.\nLet’s consider the first plot and discuss how w1 will be optimized. The process remains the same for the other parameters. Initially, the values for W and B are chosen randomly and so (w1, loss) will be randomly placed on this curve as shown by the green dot.\nNow, the goal is to reach the red dot, starting from the green dot. In other words, we need to move downhill. Looking at the slope of the curve at the green dot, we can tell that increasing w1 (moving right) will lower the loss and therefore move the green dot closer to the red one. In mathematical terms, if the gradient of the loss with respect to w1 is negative, increase w1 to move downhill and vice versa. Therefore, w1 should be updated as:\nThe equation above is known as gradient descent equation. Here, the learning_rate controls how much we want to increase or decrease w1. If the learning_rate is large, the update will be large. This could lead to w1 going past the red dot and therefore missing the optimal value. If this value is too small, it will take forever for w1 to reach the red dot. You can try experimenting with different values of learning rate to see which works the best. In general, small values like 0.01 works well for most cases.\nIn most cases, a single update is not enough to optimize these parameters; so, the process of forward-propagation and backpropagation is repeated in a loop until the loss stops reducing further. Let’s see this in action:\nAn important observation to make is that initially the green dot moves quickly and slows down as it gradually approaches the minima. The large slope (gradient) during the first few epochs (when the green dot is far from the minima) is responsible for this large update to the parameters. The gradient decreases as the green dot approaches the minima and thus the update becomes slow. The other three parameters are trained in parallel in the exact same way. Another important observation is that the shape of the curve changes with epoch. This is due to the fact that the other three parameters (w2, w3, b) are also being updated in parallel and each parameter contributes to the shape of the loss curve.\nVisualize Let’s see how the decision boundary updates in real-time as the parameters are being updated.\nThat’s all folks! If you made it till here, hats off to you! In this article, we took a visual approach to understand how machine learning works. So far, we have seen how a simple 3D to 1D mapping, f(x), can be used to fit a decision boundary (2D plane) to a linearly separable dataset (3D). We discussed how forward propagation is used to calculate the loss followed by backpropagation where gradients of the loss with respect to parameters are calculated and the parameters are updated repeatedly in a training loop.\n","permalink":"https://fred.devs.surf/blog/machine-learning-visualized/","summary":"Introduction to machine learning In the traditional hard-coded approach, we program a computer to perform a certain task. We tell it exactly what to do when it receives a certain input. In mathematical terms, this is like saying that we write the f(x) such that when users feed the input x into f(x), it gives the correct output y.\nIn machine learning, however, we have a large set of inputs x and corresponding outputs y but not the function f(x).","title":"Machine Learning - Visualized"},{"content":"Introduction In this article, we will learn how PCA can be used to compress a real-life dataset. We will be working with Labelled Faces in the Wild (LFW), a large scale dataset consisting of 13233 human-face grayscale images, each having a dimension of 64x64. It means that the data for each face is 4096 dimensional (there are 64x64 = 4096 unique values to be stored for each face). We will reduce this dimension requirement, using PCA, to just a few hundred dimensions!\nPrincipal Component Analysis (PCA) Principal component analysis (PCA) is a technique for reducing the dimensionality of datasets, exploiting the fact that the images in these datasets have something in common. For instance, in a dataset consisting of face photographs, each photograph will have facial features like eyes, nose, mouth. Instead of encoding this information pixel by pixel, we could make a template of each type of these features and then just combine these templates to generate any face in the dataset. In this approach, each template will still be 64x64 = 4096 dimensional, but since we will be reusing these templates (basis functions) to generate each face in the dataset, the number of templates required will be small. PCA does exactly this. Let’s see how!\nDataset Let’s visualize some images from the dataset. You can see that each image has a complete face, and the facial features like eyes, nose, and lips are clearly visible in each image. Now that we have our dataset ready, let’s compress it.\nCompression PCA is a 4 step process. Starting with a dataset containing n dimensions (requiring n-axes to be represented):\nStep 1: Find a new set of basis functions (naxes) where some axes contribute to most of the variance in the dataset while others contribute very little. Step 2: Arrange these axes in the decreasing order of variance contribution. Step 3: Now, pick the top k axes to be used and drop the remaining n-k axes. Step 4: Now, project the dataset onto these k axes. These steps are well explained in my previous article. After these 4 steps, the dataset will be compressed from n-dimensions to just k-dimensions (k\u0026lt;n).\nStep 1 Finding a new set of basis functions (n-axes), where some axes contribute to most of the variance in the dataset while others contribute very little, is analogous to finding the templates that we will combine later to generate faces in the dataset. A total of 4096 templates, each 4096 dimensional, will be generated. Each face in the dataset can be represented as a linear combination of these templates.\nPlease note that the scalar constants (k1, k2, …, kn) will be unique for each face.\nStep 2 Now, some of these templates contribute significantly to facial reconstruction while others contribute very little. This level of contribution can be quantified as the percentage of variance that each template contributes to the dataset. So, in this step, we will arrange these templates in the decreasing order of variance contribution (most significant…least significant).\nStep 3 Now, we will keep the top k templates and drop the remaining. But, how many templates shall we keep? If we keep more templates, our reconstructed images will closely resemble the original images but we will need more storage to store the compressed data. If we keep too few templates, our reconstructed images will look very different from the original images.\nThe best solution is to fix the percentage of variance that we want to retain in the compressed dataset and use this to determine the value of k (number of templates to keep). If we do the math, we find that to retain 99% of the variance, we need only the top 577 templates. We will save these values in an array and drop the remaining templates.\nLet’s visualize some of these selected templates.\nPlease note that each of these templates looks somewhat like a face. These are called as Eigenfaces.\nStep 4 Now, we will construct a projection matrix to project the images from the original 4096 dimensions to just 577 dimensions. The projection matrix will have a shape (4096, 577), where the templates will be the columns of the matrix.\nBefore we go ahead and compress the images, let’s take a moment to understand what we really mean by compression. Recall that the faces can be generated by a linear combination of the selected templates. As each face is unique, every face in the dataset will require a different set of constants (k1, k2, …, kn) for the linear combination.\nLet’s start with an image from the dataset and compute the constants (k1, k2, …, kn), where n = 577. These constants along with the selected 577 templates can be plugged in the equation above to reconstruct the face. This means that we only need to compute and save these 577 constants for each image. Instead of doing this image by image, we can use matrices to compute these constants for each image in the dataset at the same time.\nRecall that there are 13233 images in the dataset. The matrix compressed_images contains the 577 constants for each image in the dataset. We can now say that we have compressed our images from 4096 dimensions to just 577 dimensions while retaining 99% of the information.\nCompression Ratio Let’s calculate how much we have compressed the dataset. Recall that there are 13233 images in the dataset and each image is 64x64 dimensional. So, the total number of unique values required to store the original dataset is13233 x 64 x 64 = 54,202,368 unique values.\nAfter compression, we store 577 constants for each image. So, the total number of unique values required to store the compressed dataset is13233 x 577 = 7,635,441 unique values. But, we also need to store the templates to reconstruct the images later. Therefore, we also need to store577 x 64 x 64 = 2,363,392 unique values for the templates. Therefore, the total number of unique values required to store the compressed dataset is7,635,441 + 2,363,392 = 9,998,883 unique values.\nWe can calculate the percentage compression as:\nReconstruct the Images The compressed images are just arrays of length 577 and can’t be visualized as such. We need to reconstruct it back to 4096 dimensions to view it as an array of shape (64x64). Recall that each template has a dimension of 64x64 and that each constant is a scalar value. We can use the equation below to reconstruct any face in the dataset.\nAgain, instead of doing this image by image, we can use matrices to reconstruct the whole dataset at once, with of course a loss of 1% variance.\nLet’s look at some reconstructed faces.\nWe can see that the reconstructed images have captured most of the relevant information about the faces and the unnecessary details have been ignored. This is an added advantage of data compression, it allows us to filter unnecessary details (and even noise) present in the data.\nThat’s all folks! If you made it till here, hats off to you! In this article, we learnt how PCA can be used to compress Labelled Faces in the Wild (LFW), a large scale dataset consisting of 13233 human-face images, each having a dimension of 64x64. We compressed this dataset by over 80% while retaining 99% of the information.\nColab Notebook View my Colab Notebook for a well commented code!\n","permalink":"https://fred.devs.surf/blog/face-dataset-compression/","summary":"Introduction In this article, we will learn how PCA can be used to compress a real-life dataset. We will be working with Labelled Faces in the Wild (LFW), a large scale dataset consisting of 13233 human-face grayscale images, each having a dimension of 64x64. It means that the data for each face is 4096 dimensional (there are 64x64 = 4096 unique values to be stored for each face). We will reduce this dimension requirement, using PCA, to just a few hundred dimensions!","title":"Face Dataset Compression using PCA"},{"content":"Introduction If you have ever taken an online course on Machine Learning, you must have come across Principal Component Analysis for dimensionality reduction, or in simple terms, for compression of data. Guess what, I had taken such courses too but I never really understood the graphical significance of PCA because all I saw was matrices and equations. It took me quite a lot of time to understand this concept from various sources. So, I decided to compile it all in one place.\nIn this article, we will take a visual (graphical) approach to understand PCA and how it can be used to compress data. Basic knowledge of Linear Algebra and Matrices is assumed. If you are new to this concept, just follow along, I have tried my best to keep this as simple as possible.\nThese days, datasets containing a large number of dimensions are increasingly common and are often difficult to interpret. One example can be a database of face photographs of let’s say, 1,000,000 people. If each face photograph has a dimension of 100x100, then the data of each face is 10000 dimensional (there are 100x100 = 10,000 unique values to be stored for each face). Now, if 1 byte is required to store the information of each pixel, then 10,000 bytes are required to store 1 face. Since there are 1000 faces in the database,10,000 x 1,000,000 = 10 GB will be needed to store the dataset.\nPrincipal component analysis (PCA) is a technique for reducing the dimensionality of such datasets, exploiting the fact that the images in these datasets have something in common. For instance, in a dataset consisting of face photographs, each photograph will have facial features like eyes, nose, mouth. Instead of encoding this information pixel by pixel, we could make a template of each type of these features and then just combine these templates to generate any face in the dataset. In this approach, each template will still be 100x100 = 1000 dimensional, but since we will be reusing these templates (basis functions) to generate each face in the dataset, the number of templates required will be very small. PCA does exactly this.\nHow does PCA work? This part is going to be a bit technical, so bear with me! I will try to explain the working of PCA with a simple example. Let’s consider the data shown below containing 100 points each 2 dimensional (x \u0026amp; y coordinates is needed to represent each point).\nCurrently, we are using 2 values to represent each point. Let’s explain this situation in a more technical way. We are currently using 2 basis functions,x as (1, 0) and y as (0, 1). Each point in the dataset is represented as a weighted sum of these basis functions. For instance, point (2, 3) can be represented as 2(1, 0) + 3(0, 1) = (2, 3). If we omit either of these basis functions, we will not be able to represent the points in the dataset accurately. Therefore, both the dimensions necessary, and we can’t just drop one of them to reduce the storage requirement. This set of basis functions is actually the cartesian coordinate in 2 dimensions.\nIf we notice closely, we can very well see that the data approximates a line as shown by the red line below.\nNow, let’s rotate the coordinate system such that the x-axis lies along the red line. Then, the y-axis (green line) will be perpendicular to this red line. Let’s call these new x and y axes as a-axis and b-axis respectively. This is shown below.\nNow, if we use a and b as the new set basis functions (instead of using x and y) for this dataset, it wouldn’t be wrong to say that most of the variance in the dataset is along the a-axis. Now, if we drop the b-axis, we can still represent the points in the dataset very accurately, using just a-axis. Therefore, we now only need half as must storage to store the dataset and reconstruct it accurately. This is exactly how PCA works.\nPCA is a 4 step process. Starting with a dataset containing n dimensions (requiring n-axes to be represented):\nFind a new set of basis functions (naxes) where some axes contribute to most of the variance in the dataset while others contribute very little. Arrange these axes in the decreasing order of variance contribution. Now, pick the top k axes to be used and drop the remaining n-k axes. Now, project the dataset onto these k axes. After these 4 steps, the dataset will be compressed from n-dimensions to just k-dimensions (k\u0026lt;n).\nSteps For the sake of simplicity, let’s take the above dataset and apply PCA on that. The steps involved will be technical and basic knowledge of linear algebra is assumed.\nStep 1 Since this is a 2-dimensional dataset, n=2. The first step is to find the new set of basis functions (a \u0026amp; b). In the explanation above, we saw that the dataset had the maximum variance along a line and we manually chose that line as a-axis and the line the perpendicular to it as b-axis. In practice, we want this step to be automated.\nTo accomplish this, we can find the eigenvalues and eigenvectors of the covariance matrix of the dataset. Since the dataset is 2 dimensional, we will get 2 eigenvalues and their corresponding eigenvectors. Then, the 2 eigenvectors are two basis functions (new axes) and the two eigenvalues tell us the variance contribution of the corresponding eigenvectors. A large value of eigenvalue implies that the corresponding eigenvector (axis) contributes more towards the total variance of the dataset.\nStep 2 Now, sort the eigenvectors (axes) according to decreasing eigenvalues. Here, we can see that the eigenvalue for a-axis is much larger than that of theb-axis meaning that a-axis contributes more towards the dataset variance.\nThe percentage contribution of each axis towards the total dataset variance can be calculated as:\nThe above numbers prove that the a-axis contributes 99.7% towards the dataset variance and that we can drop the b-axis and lose just 0.28% of the variance.\nStep 3 Now, we will drop the b-axis and keep only the a-axis.\nStep 4 Now, reshape the first eigenvector (a-axis) into a 2x1 matrix, called the projection matrix. It will be used to project the original dataset of shape(100, 2) onto the new basis function (a-axis), thus compressing it to (100, 1).\nReconstruct the data Now, we can use the projection matrix to expand the data back to its original size, with of course a small loss of variance (0.28%).\nThe reconstructed data is shown below:\nPlease note that the variance along the b-axis (0.28%) is lost as evident by the above figure.\nThat’s all folks! If you made it till here, hats off to you! In this article, we took a graphical approach to understand how Principal Component Analysis works and how it can be used for data compression.\nColab Notebook View my Colab Notebook for a well commented code!\n","permalink":"https://fred.devs.surf/blog/pca-visualized/","summary":"Introduction If you have ever taken an online course on Machine Learning, you must have come across Principal Component Analysis for dimensionality reduction, or in simple terms, for compression of data. Guess what, I had taken such courses too but I never really understood the graphical significance of PCA because all I saw was matrices and equations. It took me quite a lot of time to understand this concept from various sources.","title":"Principal Component Analysis - Visualized"},{"content":"Intro In my video about How I cleared the AWS SAA Certification Exam, I shared my preparation strategy as well as tips to ace the exam. I also gave a glimpse of my revision notes that I prepared while taking the course and practice exams on Udemy. After that video was out, I got so many comments and DMs, requesting me to share my notes, but the problem was that I took these notes using a note-taking app called Obsidian which stores them in markdown format locally on my Mac. Once I\u0026rsquo;m done editing my notes, I push them to a GitHub repository to make sure I don\u0026rsquo;t lose them if my laptop breaks.\nSo, if you want to view my notes exactly like I do, you can clone my Obsidian Vault repository and download Obsidian to render it. But, this solution isn\u0026rsquo;t elegant as it would require you to download an additional software. So, I along with my college roommate, Sarthak Narayan, had been working over the past 2 weeks on the project, Obsidian Publish using GitHub Action, which would allow us to effortlessly publish our notes as a static website.\nIt is complete and I\u0026rsquo;ve used it to publish my notes at notes.arkalim.org. Working The GitHub Action spins up a Docker container which parses and converts Obsidian markdown notes into a special markdown format understood by MkDocs, an open-source static site generator. MkDocs is actually meant for preparing documentations but works well for notes too. After the markdown files have been converted, all the images in my notes are compressed to a fraction of their original size so that they can load quickly in your web browser. A static site is then built using MkDocs and then finally deployed on Netlify. All of this happens automatically using GitHub Actions. All I have to do is update my notes and push the changes to GitHub.\nFinal thoughts Having an automated way to publish your notes online with the community is a powerful way to share knowledge. This project has also made it exceedingly easy for me to refer my notes from anywhere, which is powerful when you work on a lot of systems.\nResources My Notes Obsidian Publish - GitHub Action Parser and Image Compressor MkDocs - Material Theme ","permalink":"https://fred.devs.surf/projects/obsidian-publish-github-action/","summary":"Intro In my video about How I cleared the AWS SAA Certification Exam, I shared my preparation strategy as well as tips to ace the exam. I also gave a glimpse of my revision notes that I prepared while taking the course and practice exams on Udemy. After that video was out, I got so many comments and DMs, requesting me to share my notes, but the problem was that I took these notes using a note-taking app called Obsidian which stores them in markdown format locally on my Mac.","title":"PLER Publish using GitHub Action"},{"content":"🔗 GitHub Description I like reading personal improvement and mindset change type books on Kindle e-reader. Some of these books are downloaded straight from the internet and not from the Kindle store. I take highlights during my reading which I wanted to sync to my Notion workspace. There was no existing app that could do this job, so I developed my own.\nKindle exports the highlights as a file named MyClippings.txt. The NodeJS application reads the MyClipping.txt file exported by Kindle, parses it using Regex, extracts all the highlights, book names, highlight time etc and creates a JSON. It then uses Notion API to sync these highlights to a database in my Notion workspace. The app maintains a cache (JSON) containing the number of highlights synced for each book. This allows the highlights to be synced incrementally, preventing re-syncing of old highlights.\nAfter the app was received well by the open-source community and other developers contributed to improve the app, I dockerized it to make shipping the app easier. Now, the users don’t have to install any dependency. They can just use the docker run command with the path to their clippings file along with their Notion API key and database ID. This would sync their highlights to their Notion database.\nAs a part of automation, I implemented auto build and deployment of containers on push to the master branch using GitHub Actions. If a developer raises a pull request and I merge it to the master branch, the GitHub workflow automatically builds the app and deploys it to GitHub packages repository.\n","permalink":"https://fred.devs.surf/projects/kindle-to-notion/","summary":"🔗 GitHub Description I like reading personal improvement and mindset change type books on Kindle e-reader. Some of these books are downloaded straight from the internet and not from the Kindle store. I take highlights during my reading which I wanted to sync to my Notion workspace. There was no existing app that could do this job, so I developed my own.\nKindle exports the highlights as a file named MyClippings.","title":"Kindle to Notion"},{"content":"🔗 Colab Notebook Description In this project, I implemented the paper Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. The neural network, a combination of CNN and LSTM, was trained on the MS COCO dataset and it learns to generate captions from images.\nAs the network generates the caption, word by word, the model’s gaze (attention) shifts across the image. This allows it to focus on those parts of the image which is more relevant for the next word to be generated. Furthermore, beam search is used during inference to enhance the prediction result. The network was trained in PyTorch on an Nvidia GTX 1060 graphics card for over 80 epochs.\n","permalink":"https://fred.devs.surf/projects/automated-image-captioning/","summary":"🔗 Colab Notebook Description In this project, I implemented the paper Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. The neural network, a combination of CNN and LSTM, was trained on the MS COCO dataset and it learns to generate captions from images.\nAs the network generates the caption, word by word, the model’s gaze (attention) shifts across the image. This allows it to focus on those parts of the image which is more relevant for the next word to be generated.","title":"Automated Image Captioning (Bachelor Thesis)"},{"content":"🔗 View App 🔗 GitHub Description A to-do list web application built using React that allows the user to add, remove and edit their todos. Todo lists are stored in the browser local storage. I built this app while learning React.\n","permalink":"https://fred.devs.surf/projects/todo-list-app/","summary":"🔗 View App 🔗 GitHub Description A to-do list web application built using React that allows the user to add, remove and edit their todos. Todo lists are stored in the browser local storage. I built this app while learning React.","title":"Todo List App"},{"content":"🔗 Colab Notebook 🔗 Blog Post Description In this project, I trained a neural network to localize key points on faces. Resnet-18 was used as the model with some slight modifications to the input and output layer. The model was trained on the official DLib Dataset containing 6666 images along with corresponding 68-point landmarks for each face. Additionally, I wrote a custom data preprocessing pipeline in PyTorch to increase variance in the input images to help the model generalize better. The neural network was trained for 30 epochs before it reached the optima.\nDuring inference, OpenCV Harr Cascades are used to detect faces in the input images. Detected faces are then cropped, resized to (224, 224), and fed to our trained neural network to predict landmarks in them. The predicted landmarks in the cropped faces are then overlayed on top of the original image.\n","permalink":"https://fred.devs.surf/projects/face-landmarks-detection/","summary":"🔗 Colab Notebook 🔗 Blog Post Description In this project, I trained a neural network to localize key points on faces. Resnet-18 was used as the model with some slight modifications to the input and output layer. The model was trained on the official DLib Dataset containing 6666 images along with corresponding 68-point landmarks for each face. Additionally, I wrote a custom data preprocessing pipeline in PyTorch to increase variance in the input images to help the model generalize better.","title":"Face Landmarks Detection using CNN"},{"content":"Description The aim of the project was to build goggles which could find where the user was looking (gaze), the category of object the user was looking at, and the duration of attention on that object. The goggles had 3 camera modules, one on each eye to track the pupil movement and the third one for mapping the gaze to the real world. Thresholding was used to detect the pupils and contours were used to find its centre. Various important parameters such as pupil velocity, acceleration, and fixation time were calculated for further statistical analysis. Single Shot Descriptor, with VGG16 as backbone, was used to detect the objects the user was gazing at. Additionally, a GUI was made using TkInter for ease of use.\n","permalink":"https://fred.devs.surf/projects/gaze-tracking-goggles/","summary":"Description The aim of the project was to build goggles which could find where the user was looking (gaze), the category of object the user was looking at, and the duration of attention on that object. The goggles had 3 camera modules, one on each eye to track the pupil movement and the third one for mapping the gaze to the real world. Thresholding was used to detect the pupils and contours were used to find its centre.","title":"Gaze-tracking Goggles"},{"content":"🔗 GitHub Description The aim of the project is to build an open-source quadcopter platform for research in the field of drone autonomy. Various deep learning and computer vision algorithms will be implemented on the drone including person tracking, gesture control using human pose estimation, optical flow stabilization, obstacle avoidance, and depth estimation using monocular vision. The drone uses a Pixhawk flight controller with Raspberry Pi as a companion computer. DJI Flame Wheel-450 is used for the quadcopter frame along with some custom mountings for adding additional components.\nRaspberry Pi runs a ROS node which communicates with another ROS node running on the host PC to transfer videos over Wi-Fi. To make the project open-source, easy to develop, and easily reproducible, the simulation environment setup has been dockerized using docker container. We are currently developing the algorithms and testing them in Gazebo Simulation.\n","permalink":"https://fred.devs.surf/projects/openquad/","summary":"🔗 GitHub Description The aim of the project is to build an open-source quadcopter platform for research in the field of drone autonomy. Various deep learning and computer vision algorithms will be implemented on the drone including person tracking, gesture control using human pose estimation, optical flow stabilization, obstacle avoidance, and depth estimation using monocular vision. The drone uses a Pixhawk flight controller with Raspberry Pi as a companion computer. DJI Flame Wheel-450 is used for the quadcopter frame along with some custom mountings for adding additional components.","title":"OpenQuad"},{"content":" Presented in the 4th International and 19th National Conference on Machine and Mechanisms (iNaCoMM 2019)\nPublished in the Springer 2019\n🔗 Publication Description Natural disasters like earthquakes and landslides are sudden events that cause widespread destruction and major collateral damage including loss of life. Though disasters can never be prevented, their effects on mankind can surely be reduced. In this paper, we present the design and control of SRR (Search and Reconnaissance Robot), a robot capable of traversing on all terrains and locating survivors stuck under the debris. This will assist the rescue team to focus on recovering the victims, leaving the locating task for the Robots. The unique features of the SRR above existing ATVs are active-articulation, modularity, and assisted-autonomy. Active-articulation allows the SRR to climb objects much tall than itself. Modularity allows the SRR to detach into smaller modules to enter tight spaces where the whole body can’t fit. Assisted-autonomy allows the SRR to detect the presence of objects in front and climb autonomously over them.\n","permalink":"https://fred.devs.surf/projects/search-and-reconnaissance-robot/","summary":"Presented in the 4th International and 19th National Conference on Machine and Mechanisms (iNaCoMM 2019)\nPublished in the Springer 2019\n🔗 Publication Description Natural disasters like earthquakes and landslides are sudden events that cause widespread destruction and major collateral damage including loss of life. Though disasters can never be prevented, their effects on mankind can surely be reduced. In this paper, we present the design and control of SRR (Search and Reconnaissance Robot), a robot capable of traversing on all terrains and locating survivors stuck under the debris.","title":"Search and Reconnaissance Robot"},{"content":"Description I worked on this project single-handedly during the summer break following my freshman year at NIT- Trichy. SEBART-Pro is a robot that follows a ball while balancing on two wheels. It can also recognize traffic signs and act accordingly. It has two stepper motors for precise position control and used an Arduino Nano as the microcontroller. The robot senses the tilt using an MPU-6050 (6-axis gyroscope and accelerometer) and converts the values from these sensors into angles using a Kalman Filter. It uses the PID control algorithm to balance on two wheels and a simple Convolutional Neural Network is used to recognize traffic signs.\n","permalink":"https://fred.devs.surf/projects/sebart-pro/","summary":"Description I worked on this project single-handedly during the summer break following my freshman year at NIT- Trichy. SEBART-Pro is a robot that follows a ball while balancing on two wheels. It can also recognize traffic signs and act accordingly. It has two stepper motors for precise position control and used an Arduino Nano as the microcontroller. The robot senses the tilt using an MPU-6050 (6-axis gyroscope and accelerometer) and converts the values from these sensors into angles using a Kalman Filter.","title":"SEBART-Pro"},{"content":"Description Building scalable backend infrastructure to transform the legal experience for all\n","permalink":"https://fred.devs.surf/experience/clio/","summary":"Description Building scalable backend infrastructure to transform the legal experience for all","title":"Software Developer II (Backend Infrastructure)"},{"content":"Description Developed a multi-stage LLM pipeline using Llama3 to extract key information from X-Ray report images, leading to the creation of a new product in our suite. Architected and developed a cloud-native version of the product on Kubernetes and packaged it as a Helm chart for ease of deployment, distribution and versioning. Enhanced operational efficiency by implementing GitOps at scale using ArgoCD, enabling centralized management of multiple K8s clusters, cutting down the release and software update timings by a magnitude. Established robust Kubernetes cluster monitoring and log aggregation through Grafana Agent and Grafana Cloud, allowing remote observability into customer installations. Built CI pipelines on GitHub Actions to automate building and testing of container images upon update to the product code. Automated end-to-end testing using PyTest, effectively saving over 4 hours of manual testing time per sprint. Strategically optimized the product\u0026rsquo;s infrastructure on AWS, reducing the cloud cost by over 50%. Actively engaged with potential customers as the lead developer, providing technical guidance and support to drive customer success. Revamped the entire product codebase, improving reliability and readability while fixing numerous production bugs to ensure the smooth operation of the product on customer sites. ","permalink":"https://fred.devs.surf/internships/16bit/","summary":"Description Developed a multi-stage LLM pipeline using Llama3 to extract key information from X-Ray report images, leading to the creation of a new product in our suite. Architected and developed a cloud-native version of the product on Kubernetes and packaged it as a Helm chart for ease of deployment, distribution and versioning. Enhanced operational efficiency by implementing GitOps at scale using ArgoCD, enabling centralized management of multiple K8s clusters, cutting down the release and software update timings by a magnitude.","title":"Software Engineer (LLM \u0026 Backend Infrastructure)"},{"content":"Description Developed an event-driven serverless integration pipeline using AWS Lambda and EventBridge, facilitating seamless bidirectional synchronization of customer data between Salesforce and Boomerang, thereby enhancing customer onboarding experience. Developed a Slack bot that sent interactive daily notifications to customers, facilitating direct actions such as task assignments and completions from within Slack. This streamlined operations and boosted sales by over 50%. Engineered a job tracking system to monitor asynchronous data transfer jobs via AWS AppFlow. This dramatically enhanced the reliability and transparency of data transfers between Boomerang and Salesforce. Designed and implemented a configuration-driven framework to extend the pattern-matching capability of AWS EventBridge, preventing over 1000 false invocations of AWS Lambda functions every day. ","permalink":"https://fred.devs.surf/experience/buyerassist/","summary":"Description Developed an event-driven serverless integration pipeline using AWS Lambda and EventBridge, facilitating seamless bidirectional synchronization of customer data between Salesforce and Boomerang, thereby enhancing customer onboarding experience. Developed a Slack bot that sent interactive daily notifications to customers, facilitating direct actions such as task assignments and completions from within Slack. This streamlined operations and boosted sales by over 50%. Engineered a job tracking system to monitor asynchronous data transfer jobs via AWS AppFlow.","title":"Backend Engineer"},{"content":"🔗 GitHub Description Guide: Mohammad Farid Azampour (Visiting Researcher at Chair for Computer Aided Medical Procedures, TU Munich)\nRearchitected Pix2Pix, a CGAN architecture, to generate Ultrasound (US) scans from MRI scans Addressed the challenge of structural dissimilarity between MRI and US scans due to data collection methods Developed a custom loss function combining CGAN loss and Dice Loss for improved image segmentation Implemented the loss function to encourage the generator to eliminate structural deformation in the generated US scans Leveraged remote access to TU-Munich\u0026rsquo;s cluster computers for fast model training Utilized TU-Munich\u0026rsquo;s Discourse forum for collaboration which drove the project to fruition ","permalink":"https://fred.devs.surf/experience/tumunich/","summary":"🔗 GitHub Description Guide: Mohammad Farid Azampour (Visiting Researcher at Chair for Computer Aided Medical Procedures, TU Munich)\nRearchitected Pix2Pix, a CGAN architecture, to generate Ultrasound (US) scans from MRI scans Addressed the challenge of structural dissimilarity between MRI and US scans due to data collection methods Developed a custom loss function combining CGAN loss and Dice Loss for improved image segmentation Implemented the loss function to encourage the generator to eliminate structural deformation in the generated US scans Leveraged remote access to TU-Munich\u0026rsquo;s cluster computers for fast model training Utilized TU-Munich\u0026rsquo;s Discourse forum for collaboration which drove the project to fruition ","title":"Remote Research Intern"},{"content":"Description Guide: Dr. Sripad Krishna Devalla (co-founder and CTO at OriginHealth)\nDeveloped a configuration-driven framework with data preprocessing pipeline for training deep learning models on AWS EC2 instances Standardized the training procedure for ML models, saving over 100 hours of development time for the ML team Worked on fetal head segmentation using the developed framework Gained insights into the applications of AI and Computer Vision in medical diagnosis Acquired experience in working with limited and sensitive healthcare data ","permalink":"https://fred.devs.surf/experience/origin-health/","summary":"Description Guide: Dr. Sripad Krishna Devalla (co-founder and CTO at OriginHealth)\nDeveloped a configuration-driven framework with data preprocessing pipeline for training deep learning models on AWS EC2 instances Standardized the training procedure for ML models, saving over 100 hours of development time for the ML team Worked on fetal head segmentation using the developed framework Gained insights into the applications of AI and Computer Vision in medical diagnosis Acquired experience in working with limited and sensitive healthcare data ","title":"Software Intern"},{"content":"🔗 GitHub Description Guide: Prof. Dr. Pratyush Kumar (Assistant Professor, Dept. of Computer Science, IIT Madras)\nImplemented a Convolutional Neural Network (CNN) for 6-DoF Global Pose Regression and Odometry Estimation Developed a deep learning model in TensorFlow to estimate the camera pose from consecutive monocular images The model demonstrated superior performance compared to traditional feature-based visual localization algorithms, especially in texture-less regions The neural network was later utilized for robot localization in GPS-denied environments ","permalink":"https://fred.devs.surf/experience/iit-madras/","summary":"🔗 GitHub Description Guide: Prof. Dr. Pratyush Kumar (Assistant Professor, Dept. of Computer Science, IIT Madras)\nImplemented a Convolutional Neural Network (CNN) for 6-DoF Global Pose Regression and Odometry Estimation Developed a deep learning model in TensorFlow to estimate the camera pose from consecutive monocular images The model demonstrated superior performance compared to traditional feature-based visual localization algorithms, especially in texture-less regions The neural network was later utilized for robot localization in GPS-denied environments ","title":"Computer Vision Intern"}]