-
Machine Learning for the Lazy Engineer – Vertex AI
Introduction Using the same dataset and problem that I was trying to solve in Machine Learning for the Lazy Engineer – BigQuery, I am going to show you how to come up with a solution using Vertex AI. I suggest you read through the previous post to get the context. Solution Quoting the heading from…
-
DevSecOps – Acronyms
While dealing with Security Professionals as Devops/DevSecOps person, you will encounter certain terms and acronyms. It helps to understand what they mean and what tools are available for us to satisfy the security requirements. In this blog post, I will list and describe a few terms/acronyms and tools that I have come across relating to…
-
Machine Learning for the Lazy Engineer – BigQuery
Introduction I first played with machine learning a few years ago when I built an app that applied labels to images of food. I learned a lot during that time, but I found the development setup to be too labor-intensive and my laptop got too hot during training. I got sidetracked with other cloud and…
-
BigQuery Bulk Insert using Python
To insert multiple rows at once using DML in BigQuery, you can do something like this: The above snippet inserts multiple rows into the table (table with columns: id and createdAt) in one go. You can also see that values types are being supplied (id:STRING and createdAt:TIMESTAMP). Binding the values this way (using parameters) will…
-
Row to Column in MySQL
Say you have two tables like these: And you have these values: id name 1 Kesavan 2 Madhavan user_id attribute_name attribute_value 1 GENDER M 1 PREMIUM_USER N 1 SUBSCRIBED_TO_NEWSLETTER Y 2 GENDER M 2 PREMIUM_USER Y To get a result like this: id name GENDER PREMIUM_USER SUBSCRIBED_TO_NEWSLETTER 1 Kesavan M N Y 2 Madhavan M…
-
PHP openssl_encrypt tip
Recently I had to encrypt some data in PHP and send it to a Java App, the Java app was unable to decrypt the message. I experimented with (data) padding, changing ciphers and changing the options for openssl_encrypt, but, none of those worked. It was a requirement at the Java end for the Key to…
-
AWS SSM Parameter Store IAM Policy for restricting by Path and Tag
I wanted to restrict access to some parameters based on the path and tag. Say, I have a key:/production/Param1=Value1with a tag:Application1=One I was expecting a policy like this: { “Version”: “2012-10-17”, “Statement”: [ { “Sid”: “VisualEditor0”, “Effect”: “Allow”, “Action”: [ “ssm:GetParameterHistory”, “ssm:ListTagsForResource”, “ssm:GetParametersByPath”, “ssm:GetParameters”, “ssm:GetParameter” ], “Resource”: “arn:aws:ssm:::parameter/production/*”, “Condition”: { “StringEquals”: { “ssm:resourceTag/Application1”: “One” }…
-
AWS Codecommit – PR and Commits with large files
AWS Codecommit console UI fails to display a diff when viewing PRs or Commits with large files (in my case a 7125 line XML file was part of the PR that failed to render the diff). AWS support confirmed that this is a limitation as of now and failed to provide further details (on what…
-
MySQL substring start is one based
To my surprise, MySQL SUBSTRING functions start position is one based. A start position of ‘0’ will return an empty string. If you would like to get the first 100 characters of a column named ‘my_column’ you will need to use SUBSTRING(‘my_column’, 1, 100). Using SUBSTRING(‘my_column’, 0, 100). will return an empty string. Postgres and…
-
Using Generators to flatten a JSON doc in PHP
To flatten a JSON like this: to: I used the following code: