29.11.18

Regex to extract citations and other parenthetical expressions from text

Print out parenthetical expressions using sed:
sed 's/(/\n(/g' InputFile.txt | sed 's/)/)\n/g' | grep "^(";

Remove parenthetical expressions using perl: 
perl -pe 's|(\(.*?\))||g' InputFile.txt;

26.11.18

Homebrew, Mac OS 10.11, Xcode 8, Clang, and the problem with thread_local

The version of the clang compiler that comes with Xcode 7 does not recognize the thread_local keyword.  Packages installed with Homebrew that use certain c++11 or higher commands may terminate during compilation with errors like "thread-local storage is not supported for the current target".  Xcode 8, which contains a suitable clang, cannot be run on MacOS 10.11.  Catch-22.

To work around, install a new version of gcc using Homebrew:

brew install gcc;

Now figure out which version of gcc was installed by Homebrew:

brew list gcc;

If gcc 8 was installed you will see something like: /usr/local/Cellar/gcc/8.2.0/bin/gcc-8
in the list of brew-installed gcc programs.  Now, tell Homebrew to use this newer version of gcc, instead of the old version of clang from xcode, to install your program.  As an example, I use the package poppler:

brew install --cc=gcc-8 poppler;



23.5.18

Quick create swap file on Rocks 6 cluster

swapon -s; #check existing swap
dd if=/dev/zero of=/state/partition1/swapfile bs=1024 count=200000k; #create swapfile, 200000k x 1kb = 200GB
mkswap /state/partition1/swapfile; #define swapfile
swapon /state/partition1/swapfile; #activate swapfile
cp /etc/fstab /etc/fstabORIG; #back up current fstab
echo "/state/partition1/swapfile          swap            swap    defaults        0 0" >> /etc/fstab; #make permanent

16.5.18

Quick create RAID 0 scratch drive

Assume two new drives have been installed which show up as /dev/sdb and /dev/sdc in fdisk -l.

mdadm --create --verbose /dev/md0 --level=stripe --raid-devices=2 /dev/sdb /dev/sdc;
parted -s -a optimal /dev/md0 mklabel gpt; #assign a partition tree type
parted -s -a optimal /dev/md0 mkpart primary 0% 100%; #create a single partition containing the entire disk
parted /dev/md0 print; #show some specs
mkfs.ext4 /dev/md0; #define the file system for the partition /dev/md0
mkdir /scratch; #the drive will be called 'scratch'
mount -t ext4 /dev/md0 /scratch; #mount drive for all users.
chmod -R 777 /scratch; #allow rwx access to everybody
aa=$(blkid /dev/md0 | awk '{print $2}' | sed 's/\"//g'); #get UUID of new raid array /dev/md0
cp /etc/fstab /etc/fstabORIG; #backup original fstab
echo "$aa /scratch ext4 defaults 0 0" >> /etc/fstab; #add a line to fstab to automount
umount /dev/md0; #unmount the raid array to test fstab
mount -a; #run /etc/fstab to remount
df; #make sure /dev/md0 is there

26.4.18

fsck and repair on reboot

Problem: Input/output errors observed on disk access. dmesg or cat /var/log/messages shows "Unrecovered read error".
Solution: Your hard disk is bad. At a minimum it has some bad sectors. You can try to repair it by forcing the utility fsck to run on reboot. You must be root to do this.

su;
touch /forcefsck; #the presence of this file tells system to fsck on boot
echo "-y" > /fsckoptions; #option to automatically repair errors encountered
reboot;

The system will remove these files after completing the fsck.

The Rocks version looks like this:

su;
ssh compute-0-1 'touch /forcefsck';
ssh compute-0-1 'echo -y > /fsckoptions';
ssh compute-0-1 'reboot';
exit;

If there are problems and the fsck won't complete, you may need to boot using a live disk and remove forcefsck and fsckoptions manually. This because the Rocks admin password may not work on a compute node.

Another approach is to search for bad blocks and write them to the 'bad block inode', so they will not be used in the future. This can be done interactively, e.g:
umount /dev/sda5;
e2fsck -ck /dev/sda5; #use badblocks read-only test to find bad blocks, faster.
or
e2fsck -cck /dev/sda5; #use badblocks read/write test to find bad blocks, slow.