AWS CLI Ansible Playbook on Ubuntu 14.04

Here's a simple Ansible playbook that installs the AWS CLI and all the required dependencies on Ubuntu 14.04 - should work on other versions of Ubuntu too.

Setup

  • Clone or download the jveldboom/ansible-aws-cli repo to your Ansible machine
  • Add your host information within inventories/hosts
  • Add your AWS creditentials within vars/vars.yml
Continue reading →

GoAccess Automated Reports - Last 30+ Days via Cron

First if you're not familier with GoAccess, here's a quick description from their website http://goaccess.io/:

GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly.

Typically you run goaccess on a single log file like this: goaccess -f access.log. But in our case we wanted to run it on multiple log files for a month end report. This provided some issues since we needed to be able to pass multiple files and only report within a date range.

Below is the script we came up with that's not perfect, but it's simple as does a pretty good job of what we needed. It finds all the gzipped access.log files modified within the past 35 days and pipes them into goaccess. We chose 35 days since it's possible some files may contain multiple days so 35 files should always include at least 30 days. Finally we save the report as an HTML file by the date. monthly-2015-05.html.

#!/bin/bash
DATE=$(date +'%Y.%m')
zcat `find /var/log/apache2/ -name "access.log.*.gz" -mtime -35` | goaccess > /dir/monthly-$DATE.html

Then just save the file and add the cron job to run at midnight on the first of each month.

# as a shell script
00 00 01 * * /bin/bash /dir/goaccess-monthly.sh

# or as a single cron job line
00 00 01 * *  zcat `find /var/log/apache2/ -name "access.log.*.gz" -mtime -35` | goaccess > /dir/monthly-$(date +'%Y.%m').html

Also check out the man page for more information on various options and settings.

Continue reading →

EC2 High CPU Wait and the EBS Provisioned IOPS Difference

On our small Graphite monitoring server (c4.large), we kept having very high CPU wait times. It would hover around 80% almost continuously. I could scrub back in the graph timeline to see where it started and there were no major changes made on that day. (like adding a fleet of new servers or extra data points) So I was pretty confident it was not an application issue.

Continue reading →

Heka JSON Decoder using a SandboxDecoder and Lua

First let me say if you're looking for help on Heka, check out their IRC channel. It's full of great guys that are extremely helpful! [IRC: #heka on irc.mozilla.org]

This Heka JSON encoder converts any simple key/value JSON payload into Heka fields.

Continue reading →

Drop MySQL Primary Key with Foreign Key using Laravel 4 (error 1025 & 105)

This issue is really not specifically related to Laravel. But the code below shows how to handle the issue within Laravel 4.

The issue comes from trying to delete a primary key that's also a foreign key. MySQL would spit out the following "useful" error:

SQLSTATE[HY000]: General error: 1025 Error on rename of './database/#sql-10e9_9c' to './database/table' (errno: 150) (SQL: alter table `table` drop primary key)

To solve this you just need to delete the foreign key first and then the primary key.

Schema::table('products_fulltext', function(Blueprint $table) {
   $table->dropForeign('table_field_foreign');
   $table->dropPrimary('PRIMARY');
});
Continue reading →

Laravel 4 Unable to Read Package Configuration File

I was trying to add a configuration file to an existing Laravel 4 package (revisionable) to help improve the functionality. But no matter what I tried I could not get the package to read from the src/config/config.php file.

Continue reading →

How to create a temporary storage directory that automatically deletes contents after X days (Mac)

Here's a quick way to have a directory that allows you to store files and other subdirectories for temporary usage. For example, I use this to save all my downloads and other files that I only need for the next day or two.

We first need to create a new cron job. If you're not familier with cron, it's basically instructions to the cron daemon of the general form: "run this command at this time on this date". (or for some light reading) We're going to use cron to run a command to move all the contents of a directory to the trash.

// from the terminal enter to create new crontab or open existing
crontab -e

// next press Esc+i to enter into "INSERT" mode which will allow you enter text
00  */2  *  *  *  find /path/to/temp/ -mtime +1 -exec mv {} ~/.Trash ; >/dev/null 2>&1

// then save the crontab by pressing Esc, :, w, q (write & quit)

Now let's explain what's going on with the cron job.

00 minutes (ie: 2:00,18:00)
*/2 every 2 hours
* * * every hour, day of the month, day of the week
find /path/to/temp/ finds all contents of directory
-mtime +1 where the file's make time is X days more than current date
-exec mv {} take all contents found from find comment and executes the mv (moves) command
~/.Trash ; users trash can (could be any directory though)
>/dev/null 2>&1 suppresses any output from displaying (ie fulling up your users mailbox)

Also, if you're looking for a quick way to add extra storage to your Macbook, checkout the Nifty Drives. This is what I use for my temporary storage.

Continue reading →

PHP session_start() failed no space left on device (Plesk plesk-php-cleanuper)

I kept receiving intermissive PHP warnings saying some thing like E_WARNING: session_start(): open(/var/lib/php/session/sess_ji9k4chqke3pde98a5n5m1vca5, O_RDWR) failed: No space left on device (28). Usually this would indicate that the disk where the sessions were being stored is full. The best place to start is to check if the disk where those session files are being stored is in fact NOT full.

// to check disk space usage
df -h

// should display something like this
Filesystem            Size  Used Avail Use% Mounted on
/dev/md1              4.0G  883M  3.2G  22% /
/dev/mapper/vg00-usr  4.0G  1.8G  2.0G  49% /usr
/dev/mapper/vg00-var  890G  231G  614G  28% /var
none                  5.9G  116K  5.9G   1% /tmp

In my case the disk where the sessions were being stored had plenty of space - 600GB free!

Continue reading →

Authenticate Only Certain HTTP Verbs (POST, PUT, DELETE) in Laravel 4

In working on the API for an upcoming project, I needed a way to allow all reads (GET) to be allowed without authentication but any requests (POST, PUT, or DELETE) changing the data to require authentication.

Continue reading →

New Business Cards

After server iterations, here are the new business cards to match the updated site.

Continue reading →