SMS Swimming Lap Counter in bash with Twilio

Kate’s participating in a semester long 100 mile swim challenge at CSUF through the Student Rec Center (SRC).  They’ve got a small pool you can swim laps in, and there’s a prize if you swim a certain number of miles.  To help with tracking and motivation, I wrote a super simple set of scripts for her to keep track of her miles and send her daily stats reminders.

The application is just written in bash, because bash is easy, and can be enabled for CGI in apache.  Let’s get to it!  I wanted this to be text message based, so I started with Twilio.  I purchased a new number for $1/month and set it up to handle incoming text messages with a Webhook:

So incoming messages will hit laps.sh with a query string FULL of information.  I only care about the incoming number and the message.  Let’s take a look at laps.sh:

#!/bin/bash

echo "Content-type: text/html"
echo ""

MESSAGE=$(echo $QUERY_STRING | tr '&' '\n' | egrep '^From=|^Body=' | tr '=' ' ' | awk '{ print $4 $2 }')
MSGARRAY=($MESSAGE)

## make sure it's kate
[ ${MSGARRAY[1]} == "%2B1626xxxxxxx" ] || exit 1

## make sure it's a number greater than zero
[ ${MSGARRAY[0]} -ge 1 ] || exit 1

## record value
echo -n ${MSGARRAY[0]} >> records.csv
echo -n " " >> records.csv
date +%Y-%m-%dT%H:%M:%SZ >> records.csv

## get number of days left
NOW=$(date +%s)
END="1576886400"
DAYS=$((($END-$NOW)/86400))

## get total laps swam
SWAM=$(awk '{ sum += $1 } END { print sum }' records.csv)

## lap math
TOTALLAPS="7000"
LAPSLEFT=$(($TOTALLAPS-$SWAM))
if [ $DAYS -ge 1 ]
then
  LAPSDAY=$(($LAPSLEFT/$DAYS))
else
  LAPSDAY=$LAPSLEFT
fi

RESPONSE="Added ${MSGARRAY[0]} laps to the counter!%0aThat is $LAPSLEFT laps to go!%0a$DAYS days until the end...%0aThat is $LAPSDAY laps per day if you swam every day!"

KATE="+1626xxxxxxx"
curl --silent -XPOST -d "Body=$RESPONSE" -d "From=+1657yyyyyyy" -d "To=$KATE" "https://api.twilio.com/2010-04-01/Accounts/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/Messages" -u "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx:yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy"

The script is pretty self explanatory, for the most part.  The content type stuff is required or else apache gives out 500’s.  The MESSAGE variable is just breaking apart the query string into multiple lines, grepping for the two lines I actually care about, and the following line puts them into a string array.

The “From” field needs to match Kate’s phone number, or else I just exit.  The first word of the body needs to actually be a number greater than zero, or else we exit.  Once that’s validated we store the number of laps and the date in a local csv file.

There’s 70 laps in a mile in this pool.  The ending date is in unix time (Dec 21 2019 in this case).  The rest is just a little math to build a message, and then a curl call to send a response message.

I also call a dailyreminder.sh file from a cron job.  It’s pretty much the same thing but without the query string handling and a slightly different message.  I hope she makes it to 100 miles!

Christmas Tree water level sensor and text alerting

Dry Christmas trees are a major fire hazard. I don’t have to link the videos, but they go up in flames super fast. My wife and I are pretty forgetful sometimes so I wanted to find some way of making sure that our tree didn’t go too long without a drink!

Last year I used some soil moisture sensors (but purchased in a 5-pack on ebay for cheap). I ended up using the digital outputs, suspending it in the water between the bolts on the tree stand, and hooking it up to a raspberry pi. It looked like this:
soil sensor in tree water
Well, it sort of worked.. when the water level dropped enough, it changed signals. The problem is that it wasn’t designed to be in tree-sap infested waters. So eventually it just stopped working entirely.

This year I’d like to do better. So instead of using a soil moisture sensor, I purchased a pack of DP5200 float sensors.

With a little bit of corrugated plastic (thank you election season signs) and hot glue, it was pretty easy to suspend the float at an appropriate level in the christmas tree stand:

With my new dupont connector kit it was a breeze to connect this to my Raspberry Pi.

I’m connecting the sensor to the I2C pin and ground on the Raspberry pi as it already has an internal pull-up resistor and I don’t need to add an external one for the switch.

The code consists of two new files and one modified rc.local for startup. I set up an account on Twilio to send the text messages.

First, the initialization file at /root/tree/init.sh — this simply initializes the GPIO as an input.

#!/bin/bash

echo 3 > /sys/class/gpio/export
echo in >  /sys/class/gpio/gpio3/direction

Second, the actual monitoring code at /root/tree/monitor.sh — This sends an initial state to the numbers, and then watches the state for a consistent seconds of 0 or 1, and will only send an alert if the state actually changed and was consistent for 15 full seconds. So, in the actual event of a state change from empty to full or full to empty, it will take between 16 and 30 seconds to receive an alert.

#!/bin/bash

function text {
  ## $1 = number
  ## $2 = status.  0=needs water, 1=plenty of water
  [ $2 -eq 0 ] && MESSAGE="🎄 I'M SO THIRSTY!!! 🎄"
  [ $2 -eq 1 ] && MESSAGE="🎄 I am a happy tree 🎄"
  curl -X POST -F "Body=$MESSAGE" -F "From=+1657wwwxxxx" -F "To=$1" "https://api.twilio.com/2010-04-01/Accounts/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/Messages" -u "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx:yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy"
}

KATE="+1626yyyzzzz"
RAY="+1626xxxyyyy"
WAITS=$(seq 1 15)

OLDSTATE=$(cat /sys/class/gpio/gpio3/value)

text $RAY $OLDSTATE
text $KATE $OLDSTATE

while true
do

  COUNT=0
  for i in $WAITS
  do
    # 0 needs water, 1 has enough water
    STATE=$(cat /sys/class/gpio/gpio3/value)
    let "COUNT = COUNT + STATE"
    sleep 1
    echo $COUNT
  done

  # only message on consistent change
  if ! ((COUNT % 15))
  then
    if [ $STATE -ne $OLDSTATE ]
    then
      OLDSTATE=$STATE
      text $RAY $STATE
      text $KATE $STATE
    fi
  fi

sleep 1
done

Finally, a simple modification to /etc/rc.local, add these two lines before “exit 0”

/root/tree/init.sh
/root/tree/monitor/sh &

That’s it! Super easy and fun, and no tree fires!

Shrinking CentOS 7 EBS boot volume on AWS

I’m really bummed when I provision a boot volume that’s too large..

[rgibson@centos7 ~]$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       50G  6.5G   44G  13% /

At the current GP2 price of $0.05/gb, I’m paying three or four dollars more per month than I really should be. It’s really easy to make boot volumes bigger, but there’s not a lot of instruction out there on making them smaller. This method also has the advantage of removing the marketplace codes on the volume, making it easier to attach to a running instance later.

Shutting down the instance you want to shrink the boot volume for. You’ll need a second instance of a different OS, or at least one from a different AMI, as we can’t reliably boot up if there are multiple volumes attached with the same UUID. I used an Ubuntu instance (16.04).

First, detach the root volume you want to shrink from the original instance:

Detach original root volume

(Side note, this would be a good time to take a snapshot of this volume just in case you make a mistake in this process, such as wiping the partition table or running mkfs on the wrong volume…)

Attach it as a volume on your powered-off Ubuntu instance. I used sdf:

Attach volume to sdf on Ubuntu instance

Create a smaller replacement volume in the correct availability zone. This would be a good time to encrypt it as well…

Create new smaller volume. Encrypt it.

Attach this volume as sdg to the Ubuntu instance, and power it on. SSH in and create a new empty partition on the new volume. Pay attention that you’re working on the NEW volume and not messing with your source volume. Since we attached it as sdg it will be visible as /dev/xvdg:

ubuntu@ubuntu:~$ sudo -i 
root@ubuntu:~# fdisk /dev/xvdg 
 
Welcome to fdisk (util-linux 2.27.1).                                                                                                                                  
Changes will remain in memory only, until you decide to write them.                                                                                                    
Be careful before using the write command. 
 
Device does not contain a recognized partition table. 
Created a new DOS disklabel with disk identifier 0x1db62587. 
 
Command (m for help): n 
Partition type 
   p   primary (0 primary, 0 extended, 4 free) 
   e   extended (container for logical partitions) 
Select (default p): p 
Partition number (1-4, default 1):  
First sector (2048-41943039, default 2048):  
Last sector, +sectors or +size{K,M,G,T,P} (2048-41943039, default 41943039):  
 
Created a new partition 1 of type 'Linux' and of size 20 GiB. 
 
Command (m for help): w 
The partition table has been altered. 
Calling ioctl() to re-read partition table. 
Syncing disks. 
 
root@ubuntu:~#

Format the new partition as XFS:

Edit: If you use an OS that’s generations newer than the one you’re fixing, you could get into a bind here if you’re using a version of mkfs.xfs that writes a higher version signature than your original kernel supports.  Maybe double check this now??

root@ubuntu:~# mkfs.xfs /dev/xvdg1
meta-data=/dev/xvdg1             isize=512    agcount=4, agsize=1310656 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=0
data     =                       bsize=4096   blocks=5242624, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
root@ubuntu:~#

Make mount points for the original and new volumes, and mount them. I mounted the original drive read only:

root@ubuntu:~# mkdir /mnt/xvdf1
root@ubuntu:~# mkdir /mnt/xvdg1
root@ubuntu:~# mount -o ro /dev/xvdf1 /mnt/xvdf1
root@ubuntu:~# mount /dev/xvdg1 /mnt/xvdg1

Now we need to clone the filesystem from the old drive to the new drive. Originally I was doing this with rsync, but due to sparse file handling, the sizes didn’t match up exactly on the destination drive, so now I use a fancy piped tar command and it comes out perfect. We have to start out in the source directory for this to work right:

root@ubuntu:~# cd /mnt/xvdf1
root@ubuntu:/mnt/xvdf1# tar cSf - . | cat | (cd ../xvdg1/ && tar xSBf -)

There will be some warnings from tar about ‘socket ignored’ and this is expected.

After all of the files are copied, we need to fix the volume UUID. While we could update the UUID of the new drive to match the UUID of the old drive, there are advantages to using a new UUID. For example, if you had to perform an offline fix on this drive in the future, you could attach it to one of your other running CentOS instances and mount it. However, if you keep the old UUID you’ll have difficulty attaching it to another CentOS instance because the root UUIDs will match and the OS will get confused. Either way, we need to get a list of the UUID’s on the attached disks. This is done with the blkid command:

root@ubuntu:/mnt/xvdf1# cd ../xvdg1/
root@ubuntu:/mnt/xvdg1# blkid
/dev/xvda1: LABEL="cloudimg-rootfs" UUID="567ab888-a3b5-43d4-a92a-f594e8653924" TYPE="ext4" PARTUUID="1a7d4c6a-01"
/dev/xvdf1: UUID="0f790447-ebef-4ca0-b229-d0aa1985d57f" TYPE="xfs" PARTUUID="000aec37-01"
/dev/xvdg1: UUID="6094350f-7d18-4256-b52e-6dbf5f196219" TYPE="xfs" PARTUUID="1db62587-01"

xvda1 is the ubuntu system, so forget about that UUID. xvdf1 has the original UUID that came with the CentOS AMI. xvdg1’s UUID was created when we ran mkfs.xfs. While you *could* use xfs_admin to assign the UUID from xvdf1 onto xvdg1 and skip to installing grub, I think it’s better to put the new UUID in all the places it need to go, so that’s what I’m going to describe next. There are four files to modify, and we can use sed to perform the editing on each one of them. The format of the command is:

# sed -i -e 's/old_UUID/new_UUID/g' /path/to/file

We’re already in /mnt/xvdg1/ from when we ran the tar pipe, so the file paths are relative to this point:

root@ubuntu:/mnt/xvdg1# sed -i -e 's/0f790447-ebef-4ca0-b229-d0aa1985d57f/6094350f-7d18-4256-b52e-6dbf5f196219/g' etc/fstab
root@ubuntu:/mnt/xvdg1# sed -i -e 's/0f790447-ebef-4ca0-b229-d0aa1985d57f/6094350f-7d18-4256-b52e-6dbf5f196219/g' boot/grub2/grub.cfg
root@ubuntu:/mnt/xvdg1# sed -i -e 's/0f790447-ebef-4ca0-b229-d0aa1985d57f/6094350f-7d18-4256-b52e-6dbf5f196219/g' boot/grub/grub.conf 
root@ubuntu:/mnt/xvdg1# sed -i -e 's/0f790447-ebef-4ca0-b229-d0aa1985d57f/6094350f-7d18-4256-b52e-6dbf5f196219/g' boot/grub/menu.lst

I really don’t think the last two files are necessary but the old UUID is in there so I changed it anyway. The last step is to chroot into the disk and to install grub. In order for that to work, we need to bind mount some places from the running Ubuntu. Grub needs to be installed on the new boot drive (/dev/xvdg), NOT the partition (/dev/xvdg1).

root@ubuntu:/mnt/xvdg1# mount --bind /dev dev
root@ubuntu:/mnt/xvdg1# mount --bind /proc proc
root@ubuntu:/mnt/xvdg1# mount --bind /sys sys
root@ubuntu:/mnt/xvdg1# chroot .
[root@centos /]# grub2-install /dev/xvdg
Installing for i386-pc platform.
Installation finished. No error reported.
[root@centos /]# exit
exit
root@ubuntu:/mnt/xvdg1#

At this point, we are just about done. Shut down the ubuntu instance, detach the new drive we created, and attach it to the original instance as /dev/sda1.

Attach replacement as /dev/sda1

That’s it! Start your original instance, log in and verify the size:

[rgibson@centos ~]$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       20G  6.5G   13G  33% /

If your instance is unreachable or you otherwise can’t log in, diagnose it with AWS’s “view instance screenshot” which is probably the coolest feature they’ve added to EC2 lately. Delete your old volume and any snapshots when you’re comfortable doing so.

Happy shrinking!