Puppet Duplicate Resources with PuppetDB

Written by

If you’re using puppet and exported resources and get an error while running the puppet agent showing

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: A duplicate resource was found while collecting exported resources, with the type and title CMD[Title] on node HOSTNAME
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

Then you’ve got a duplicate resource in the puppetdb which typically happens when you export resources to be imported else which has already happened, usually because you’ve rebuilt the host and not deactivated it prior

puppet node deactivate HOSTNAME

To resolve this you can remove the previous exports from the database. I’ve used a select statement first to check what is returned


su - postgres
psql puppetdb
> SELECT * FROM catalog_resources WHERE title LIKE '%Title%';
# DELETE FROM catalog_resources WHERE title LIKE '%Title%';

Built in PuppetDB (HSQLDB):

systemctl stop puppetdb
cd /var/lib/puppetdb/db/
cat -n db.script | grep <Title>
sed -i.bak -e '<n>d' db.script
systemctl start puppetdb

Please replace <Title> with the title of your export and replace <n> with the line number that is returned from the cat command.

Some of this information is from Chris Cowley and he’s done a better job and writing about it.

Linux ZFS Quotas and my hacked solution

Written by

ZFS in linux doesn’t support quotas, this is a pain in the arse.
After deploying ZFS on our various NFS servers, we were hitting issues where users can’t check their ZFS quotas and specifically on their home space directories.

After a frustrating amount of time researching, I found that there is a Github PR/Merge awaiting for the support and not wanting to compile my own version of ZFS for production gear, I wrote a little set of scripts to hack around the issue for the time being.

You can find a version of the scripts on my github grid-scripts and the latest at github zfs-quota.

Just pop the server script on your ZFS server and have cron run it every x minutes along with piping the stdout to a shared location (we’re using NFS mounted on /mnt/home and then the client script needs to be on all client machines (we used puppet to distribute it). The you should have ZFS Quotas for your end users.

Quota Report

ZFS Quota Output

Its a very quick and dirty script hacked together to solve the problem quickly, so your mileage my vary. If you have any improvements or suggestions, please don’t hesitate to contact me.

ARGUS Pool User to Certificate WebUI

Written by

As previously mentioned, I have written a nasty bash script to get the user certificate DN for a given username in ARGUS.

This is a step further, it is a WebUI that is seeded by data updated via a script in a cronjob on ARGUS, it requires a shared filesystem (in this case, /mt/admin or it requires ARGUS to have PHP and a WebServer.

The latest files are on github.

It depends on the following:


The bash/shell script is run on argus to produce an output, every 30minutes should suffice.

The web files only need access to the output file created by the shell script along with a basic webserver running php and php-ldap.

The search box at the top, if it correctly loads all required javascript allows users to search the table beneath, including partial searching.

A table displaying ARGUS Pool users and Certificates


ARGUS Pool User to Certificate

Written by

This is a nasty short bash script to get the certificate DN for a given grid pool user from ARGUS grid map directory.

Get the script from github.

To use it just do the following:

argus_user_cert.sh <username>

to return

User pheno071 Certificate Details:
%2fc%3duk%2fo%3descience%2fou%3ddurham%2fl%3descience%2fcn%3dadam%20boutcher:pheno pheno071

The script was written very quickly and I don’t recommend using it in production or as part of a larger system, so use it at your own risk.

Simple Disk Pool Manager (DPM) Nagios Test

Written by

There are several DPM testing tools and a suite of DPM nagios plugins (see the monitoring docs) but these utilise NRPE and are relatively complex compared the the quick and useful DPM-Tester.py

This is a script that wraps that test tool for nagios; the original test script doesn’t exit with any exit codes, so it is grep’ing for the word fail and counting the number of lines, it is also checking to see if the script seg faults as it does randomly seg fault if a proxy isn’t set.

This assumes you have half a clue of what you’re doing and are running the script on the nagios host, a host certificate with the appropriate permissions in DPM and a passwordless key.

The latest version of this script can be found on github


# Nagios Plugin Wrapper for checking DPM
# Adam Boutcher - May 2017 - GPLv3
#   I would suggest reading this script and implementing your own version of it.
#   Put your host certificate & key in /etc/nagios/ and 400 it to nagios.
#   Add a Grid Map for this host certiciate to your DPM Test PATH on your DPM Server.
#   requires dpm-tester.py

if [[ -z "$1" ]]; then
echo "No Arguments Supplied"
echo "Check --usage for usaged details."
exit 1
elif [ $1 = "-u" ] || [ $1 = "--help" ] || [ $1 = "--usage" ]; then
echo "check_dpm Super Simple DPM tester for Nagios - I personally wouldn't use it."
echo "Usage:"
echo "-h --host Hostname"
echo "-t --test Test [davs, root, gsiftp, combined]"
echo "-p --path Path to test"
echo " --help Same as -u --usage"
echo "-u --usage This screen"
exit 0
while [[ $# -gt 1 ]]

case $key in
echo "Wrong Arguments Supplied."
echo "Check --usage for usaged details."
exit 1
# Get a Proxy from host cert - chmod 400 these files and own it by your nagios user.
# Only renew if it's expired
export X509_USER_CERT=/etc/nagios/hostcert.pem
export X509_USER_KEY=/etc/nagsu ios/hostkey.pem
SECPROX=$(arcproxy -i validityEnd)
SECNOW=$(date +%s --date "30 seconds")
if [ $SECPROX -le $SECNOW ]; then
arcproxy >/dev/null 2>&1

DIFF=$(dpm-tester.py --host ${DHOST} --path ${DPATH} --tests ${DTEST} --cleanup | grep -i FAIL | wc -l)

# Test for the number of FAIL lines counted OR segfault Exit code (dpm-tester.py hasn't got exit codes implemented)
if [[ "$DIFF" > "0" || "$?" > "0" ]]; then
OUTPUT=$(dpm-tester.py --host ${DHOST} --path ${DPATH} --tests ${DTEST} --cleanup | tail -n1)
exit 2;
echo "OK - DPM ${DTEST}";
exit 0;