Kubernetes Cheatsheet
The get
parameter is a powerful way of discovering your
kubenetes resources. You can use it to query: * namespace * pod * node *
deployment * service * replicasets
$ kubectl get nodes
$ kubectl get ns # ns is an abreviation for namespace
$ kubectl get pods -n kube-system
The create
command can do just that for:
- service
- cronjob
- deployment
- job
- namespace (or ns)
$ kubectl create ns hello-world
$ kubectl create cronjob my-cronjob --image=alpine --schedule="*/15 * * * *" -- echo "hi there"
You can also use cj
as an abreviation for
cronjob
$ kubectl create cj my-cronjob --image=alpine --schedule="*/15 * * * *" -- echo "hi there"
The edit
parameter allows you to update resources:
$ kubectr edit my-cronjob
The delete
parameter allows you to remove resources:
$ kubectl delete cronjob my-cronjob
The apply
parameter allows you to apply configurations
from files
$ kubectl apply -f jenkins.yaml
The describe
parameter provides details of your
resources which could be:
- nodes
- pods
- services
- deployments
- replicasets
- cronjobs
$ kubectl describe cronjob my-cronjob
The logs
parameter displays the contents of the
resource’s log:
$ kubectl logs my-resource -n charts
The exec
parameter allows you to exec into a
container:
$ kubectl exec -it my-resource -n charts -- /bin/bash
The cp
parameter lets you copy files and directories to
and from containers:
$ kubectl cp file1.txt my-resource:file1.txt
Tags: cli, kubernetes, cheatsheet, motd
Extract a Single File from a Tarball
Suppose I have a tarball (.tar.gz file) which is large and I only want to extract a specific file from it. If I know the name of the file all I have to do is pass the file’s relative path that it is stored under to the command line.
Here is an example of the error you will get if you pass the incorrect file specification:
$ tar zxvf dirtree-tarball.tar.gz file-7-30003.txt
tar: file-7-30003.txt: Not found in archive
Since I don’t have the full path, I can just search for it:
$ tar tf dirtree-tarball.tar.gz | grep 'file-7-30003.txt'
./dir_2/file-7-30003.txt
Now I can pass the full path and extract the file:
$ tar zxvf dirtree-tarball.tar.gz ./dir_2/file-7-30003.txt
./dir_2/file-7-30003.txt
$ ls
dir_2 dirtree-tarball.tar.gz
$ tree
.
├── dir_2
│ └── file-7-30003.txt
└── dirtree-tarball.tar.gz
1 directory, 2 files
Note that it extracts it to the same directory tree but it will only extract the file(s) specified on the command line.
Updating My Home Lab using Ansible
I have a variety of Raspberry Pis that I use for various tasks like my Tiny-Tiny RSS server, Gitea server, and Calibre server among other things. In order to keep them updated I use Ansible.
My update script is fairly simple:
$ cat update-pis.sh
#!/bin/bash
ansible-playbook ./playbooks/apt.yml --user memyselfandi \
--ask-pass --ask-become-pass -i ./inventory/hosts $@
The YAML playbook is likewise very simple:
$ cat ./playbooks/apt.yml
- hosts: "*"
become: yes
tasks:
- name: Update System Package Cache (apt)
apt:
update_cache: yes
upgrade: 'yes'
- name: Upgrade System Packages (apt)
apt: upgrade=full
- name: Remove unused dependencies
apt: autoremove=true
- name: Check if reboot is required
shell: "[ -f /var/run/reboot-required ]"
failed_when: False
register: reboot_required
changed_when: reboot_required.rc == 0
notify: reboot
handlers:
- name: reboot
command: /sbin/reboot
Although I can run this in a cronjob, I tend to run it manually (for now). I’m thinking about doing some major revisions to my Pi configuration anyway. Stay tuned for more on that subject.
List All MACs on the Local Network
Per the manpage:
The
arp
tool manipulates or displays the kernel’s IPv4 network neighbor cache. It can add entries to the table, delete one or display the current content. ARP stands for Address Resolution Protocol, which is used to find the media access control address of a network neighbor for a given IPv4 Address.
You can use the arp
command to list all of the devices
on the local network. It is often useful to find the identities of
hidden devices on your network. For example, if you just
plugged a Raspberry Pi into your local network and need to find its IP
address in order to connect to it via SSH.
# arp -i eth0 -a
? (192.168.1.90) at 42:b9:72:xx:xx:x0 [ether] on eth0
? (192.168.1.70) at 54:04:a6:xx:xx:xd [ether] on eth0
Fios_Quantum_Gateway.fios-router.home (192.168.1.1) at 20:c0:47:xx:xx:x1 [ether] on eth0
? (192.168.1.99) at 34:64:a9:xx:xx:xd [ether] on eth0
? (192.168.1.60) at dc:a6:32:xx:xx:x3 [ether] on eth0
(I’ve masked some of the fields above)
Tags: cli, arp, networking, motd
KVM: Configure libvirt Network
You can update the network configuration for your KVM installation
using the command line using the virsh
command.
To list all of the available network enter the following command. The
--all
will is used to include the inactive networks:
# virsh net-list --all
Name State Autostart
-----------------------------------------
default active yes
NattedNetwork active yes
Then, edit the network you wish to update:
# EDITOR="vi" virsh net-edit NattedNetwork
Add host configuration(s) or whatever changes you wish to make to XML file:
<network>
<name>NattedNetwork</name>
<uuid>8483028d-667b-47e7-9a8e-f269783a8246</uuid>
<forward mode='nat'/>
<bridge name='virbr0' stp='on' delay='0'/>
<mac address='52:54:00:ad:b9:ed'/>
<ip address='192.168.122.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.122.2' end='192.168.122.254'/>
</dhcp>
</ip>
</network>
Once you’re done, restart the network for the changes to take effect:
# virsh net-destroy NattedNetwork
# virsh net-start NattedNetwork
Tags: cli, libvirt, kvm, network, motd
Install sbopkg on Slackware
Sbopkg is a command‐line and dialog‐based tool to interact with the SlackBuilds.org repository, a collection of third‐party SlackBuild scripts to build Slackware packages.
Here are the steps for installing the latest version of sbopkg with Slackware.
Download the latest Slackware package from
https://www.sbopkg.org/downloads.php
using whatever method you find most convenient. At the time of this writing the latest version ishttps://github.com/sbopkg/sbopkg/releases/download/0.38.2/sbopkg-0.38.2-noarch-1_wsr.tgz
. I will use that version in the commands below. If the version has changed, you would use the new filename in the commands. You can also usewget <packagename>
to download it directly.Open your favorite terminal and
su
to root.Change to the directory where you downloaded the package.
Run the command
installpkg sbopkg-0.38.2-noarch-1_wsr.tgz
If all goes well, sbopkg will now be installed.
To run sbopkg, open your favorite terminal,
su -
to root, and type sbopkg
.
As usual, see the man
or info
page for more
information.
Tags: cli, slackware, sbopkg, motd
Copying a Directory Tree Recursively Using tar
You can use tar
to copy a directory tree to another
directory in one shot along with preserving ownership,
permissions and timestamps. This also avoids making an intermediary
tarfile which may cause problems if the size of the file copy
is large and the storage resources are low. Just cd
to the
top of the directory that you want to copy and begin.
Let’s assume that you want to copy the contents of the source directory to a target directory:
$ cd /path/to/source
$ tar cf - * | (cd /target; tar xfp - )
Removing Blank Spaces from a Group of File Names
This set of commands iterate over each file in the current directory
and will replace any blank spaces in a filename with an underscore
(_
).
First, we’ll create a set of sample files:
$ for i in {0..9}; do touch "file-${i} (number-${i}).txt"; done
$ ls
file-0\ (number-0).txt file-4\ (number-4).txt file-8\ (number-8).txt
file-1\ (number-1).txt file-5\ (number-5).txt file-9\ (number-9).txt
file-2\ (number-2).txt file-6\ (number-6).txt
file-3\ (number-3).txt file-7\ (number-7).txt
Now, we’ll execute the set of commands:
$ for f in *\ *; do mv "$f" "${f// /_}"; done
And now you can see that the blanks have been replaced.
$ ls
file-0_(number-0).txt file-4_(number-4).txt file-8_(number-8).txt
file-1_(number-1).txt file-5_(number-5).txt file-9_(number-9).txt
file-2_(number-2).txt file-6_(number-6).txt
file-3_(number-3).txt file-7_(number-7).txt
Remove Parentheses from File Names
This script will rename Windows backup files by removing the ’
(date)’ from the filename. It will remove the parentheses and everything
between them from the file name, so that a file with the name
file-1 (2023-09-10).txt
will be renamed to
file-1.txt
. Note that since the data between the
parentheses will be removed, the resulting file name must be unique for
this script to prevent data from being overwritten. Other than that it
should work for any fileset that meets the beforementioned
constraints.
First, let’s create some sample files:
$ for i in {0..9}; do touch "file-${i} (number-${i}).txt"; done
$ ls
file-0\ (number-0).txt file-4\ (number-4).txt file-8\ (number-8).txt
file-1\ (number-1).txt file-5\ (number-5).txt file-9\ (number-9).txt
file-2\ (number-2).txt file-6\ (number-6).txt
file-3\ (number-3).txt file-7\ (number-7).txt
Now, we’ll execute the script on the sample files:
$ for f in *; do n=$(echo $f | sed "s/[ (][^)]*[)]//g");mv "${f}" "${n}"; done
If we look at the directory listing now we see that the file names are updated:
$ ls
file-0.txt file-2.txt file-4.txt file-6.txt file-8.txt
file-1.txt file-3.txt file-5.txt file-7.txt file-9.txt
Swapping the CMD and Control Keys on a Macbook Running Linux
I find it annoying not having a CTRL key available for both hands so I tend to swap the Command and CTRL key settings on my 2012 “Slackbook”.
- Launch a terminal
- Edit the X11 Keyboard Extension
# vim /usr/share/X11/xkb/symbols/pc
- Make the following changes. Ensure your file looks like this:
key <LCTL> { [ Super_L ] };
key <LWIN> { [ Control_L ] };
...
key <RCTL> { [ Super_R ] };
key <RWIN> { [ Control_R ] };
- Clear xkb’s cache
# rm -rf /var/lib/xkb/*
- If the keys are not swapped after Step 4, restart your session or your computer.
Tags: cli, slackware, xkb, motd
IP Address Blocks
There are 4 classes of IPv4 networks:
A Class A network can have 16,777,214 computers on it. It has an IP address that starts with a number in the range 0 to 126. 127 is reserved for the local loopback. (Only 127.0.0.1 is ever used although this may change in the future).
A Class B network can have 65,534 computers on it. It has an IP address that starts in the range of 128 to 191.
A Class C network can have 254 computers on it. This network is often used for many home networks and most company networks. It has an IP address that starts with a number from 192 to 223.
Class D (224–239) is used for multicast and Class E (240–254) is reserved.
Within the above classes are sets of private network addresses:
- Any address that starts with 10
- Any address that starts 172 and where the second number is 16–31.
- Any address that starts with 192.168
Routers will generally discard any traffic to private network addresses. So,in order to talk to other devices, your router does something called Network Address Translation or NATtin. You can read more about Network Address Translation and how it works here.
Tags: cli, networks, ip-address, motd
Vim: More Copying Tips
Blocks of lines can be copied in Vim using ranges, i.e., two numbers separated by a comma indicating the start and end of the block of lines.
Some Examples
To copy line 20-25, use:
:20,25yank
To copy five lines just above the cursor:
:-4,.yank
where the period (.) indicates the current line.
To copy five lines just below the cursor:
:.,+4yank
Tags: cli, vim, copy, ranges, motd
Log File Maintenance and Cleanup
Log files sometimes take up a lot of disk space. Some applications have internal processes that run periodically to manage them and some do not. Often, this becomes a problems after a while as the logs consume all of your partition space.
You can manage these files yourself with a simple script running in a cronjob (or systemd timers if you’re so inclined) if they have a common naming convention and you have the proper access.
Let’s say you have an application called myapp that keeps
it’s logs in a directory called /opt/myapp/logs
and those
files all end with a .log
file extension.
cat >logmanage.sh <<"EOF"
#!/bin/sh
LOGDIR="/opt/myapp/logs"
# Compress all of the files older than a day
find ${LOGDIR} -name '*.log' -mtime +0 -exec compress {} \;
# Purge all of the logs older than a week
find ${LOGDIR} -name '*.Z' -mtime +7 -exec rm -f {} \;
EOF
These two commands will compress those files that are more than older than a day and remove the compressed files after a week.
Add a crontab entry to run this everyday and you’re all set.
Tags: cli, sysadmin, logs, find, motd
Vim: How to Copy & Paste
Most Vim tutorials and cheatsheets out there will cover how to copy and paste. Pasting is the easy part. Let’s talk about copying!
Copying
To copy the current line (line in which the cursor is placed), simply
press double lower y
(yy
) or single upper case
y
(Y
).
yy
or Y
: copies/yanks the current line,
including the newline character at the end of the line.
To copy a specific number of lines under the cursor (beginning from
the current location of the cursor) simply precede the command with the
number of lines that you wish to copy, i.e., Nyy
(where N
is the number of lines).
You can also use the following operation to copy two lines:
:.,+yank
where the dot (.) refers to the current line and plus sign (+) means plus one line below the cursor.
So, in order to use this syntax to copy 3 lines, use 3yy
or :.,+2yank
will work.
You can also copy a specific number of lines above the
cursor From the current cursor location use yN-1k
, where
N-1
is the number of lines you want to copy. So, to copy
two lines from the current position of the cursor, press (in normal
mode, or escape mode), y2-1k
, or y1k
.
To copy 5 lines above the current cursor position:
y4k
You can also use other operations. for example, to copy only the fifth line just above the cursor:
Copying & Pasting
:-4yank
You can can copy lines and paste them right above the cursor in one
shot using the operation :Nt-
, where N
is the
line number you want to copy.
You can can copy specific line number and paste right below the
cursor directly using the operation :Nt.
, where
N
is the line number you want to copy.
For example, to copy line two and paste just above the current cursor position;
:2t-
To copy line two and paste just below the current cursor position;
:2t.
The above commands also work with a range of lines rather than a
single line, such as :4,7t.
will yank lines 4-7 and paste
them below the cursor.
Pasting
p
or P
: pastes one or more lines, basically
whatever is in the buffer, under or above the cursor respectively.
Format Disk using exFat on Command Line
You may not always be working on a Linux system using a GUI, like a server or a system with very low resources. Sometimes you may need to format a disk to exFat using the Command Line. It’s not really that difficult.
The first thing you need to do is to know which device is to be
formatted so before you connect the disk or USB make sure you run the
lsblk
command before and after doing so. This way you will
be certain not to format the wrong disk by mistake and cause a data
loss.
Run lsblk
before inserting the disk:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 232.9G 0 disk
├─sda1 8:1 0 630M 0 part /boot/efi
├─sda2 8:2 0 216.3G 0 part /
└─sda3 8:3 0 15.9G 0 part [SWAP]
sdb 8:16 0 1.4T 0 disk
├─sdb1 8:17 0 630M 0 part
├─sdb2 8:18 0 244.1G 0 part /mnt/lfs
├─sdb3 8:19 0 1.1T 0 part /home
└─sdb4 8:20 0 15.6G 0 part
sdc 8:32 1 0B 0 disk
sr0 11:0 1 1024M 0 rom
Then insert the disk and run lsblk
again:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 232.9G 0 disk
├─sda1 8:1 0 630M 0 part /boot/efi
├─sda2 8:2 0 216.3G 0 part /
└─sda3 8:3 0 15.9G 0 part [SWAP]
sdb 8:16 0 1.4T 0 disk
├─sdb1 8:17 0 630M 0 part
├─sdb2 8:18 0 244.1G 0 part /mnt/lfs
├─sdb3 8:19 0 1.1T 0 part /home
└─sdb4 8:20 0 15.6G 0 part
sdc 8:32 1 0B 0 disk
sdd 8:48 1 979.8M 0 disk <--+
├─sdd1 8:49 1 50M 0 part <--|--New device
└─sdd2 8:50 1 928.8M 0 part <--+
sr0 11:0 1 1024M 0 rom
Compare the output and you’ll see that in this case, the device
inserted was /dev/sdd
.
Armed with that information, we can now begin our task. Start by
invoking parted
with the -a optimal
option to
specify the optimal alignment type:
root@slacker:~# parted -a optimal /dev/sdd
GNU Parted 3.4
Using /dev/sdd
Welcome to GNU Parted! Type 'help' to view a list of commands.
Next, set the partition table to gpt:
(parted) mktable gpt
Warning: The existing disk label on /dev/sdd will be destroyed and all data on
this disk will be lost. Do you want to continue?
Yes/No? Yes
Now, create a new partition table of exFAT type:
(parted) mkpart exFAT 0% -1
Set the flag for the first partition to msftdata:
(parted) set 1 msftdata on
(parted) align-check opt 1
1 aligned
(parted) quit
Information: You may need to update /etc/fstab.
Now you should be done. Let’s print out the partition to make sure everything is OK:
# parted /dev/sdd print
Model: SanDisk U3 Cruzer Micro (scsi)
Disk /dev/sdd: 1027MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 1027MB 1026MB fat32 exFAT msftdata
Now we can format it:
# mkfs.exfat -n exFAT /dev/sdd1
exfatprogs version : 1.1.3
Creating exFAT filesystem(/dev/sdd1, cluster size=32768)
Writing volume boot record: done
Writing backup volume boot record: done
Fat table creation: done
Allocation bitmap creation: done
Upcase table creation: done
Writing root directory entry: done
Synchronizing...
exFAT format complete!
#
Tags: cli, exfat, parted, motd
Find Your External IP Address
It’s easy to find your internal IP address my using tools like ifconfig or ip a but to find your external IP address (the one that connects you to the outside world) you must use other means. Here are 3 simple commands that you can use to do just that:
$ curl ifcfg.me
72.76.yyy.xxx
$ curl icanhazip.com
72.76.yyy.xxx
$ nslookup myip.opendns.com. resolver1.opendns.com
Server: resolver1.opendns.com
Address: 208.67.222.222#53
Non-authoritative answer:
Name: myip.opendns.com
Address: 72.76.yyy.xxx
Tags: cli, network, curl, nslookup, motd
Vim: Underlining a Line
Of course, Vim is just a text editor but you can
underline a line using a set of keystrokes that copies the
string to be underlined to the next line and then replaces the string
with hyphens using this set of Vim commands: YpVr-
So if you have a line:
This Is My Page Title
just position the cursor at the start of that line and enter the abovementioned commands and it will turn it into:
This Is My Page Title
---------------------
This would save you a few keystrokes if you use Vim for creating a lot of text documentation and use hyphens a lot in your section headers.
Finding Duplicate Files in a Directory Tree
Sometimes I need to find all of the duplicate files in a directory tree. I have this issue all of the time when I move my music collection. Here is a nifty script to sort these things out:
#!/bin/bash
find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 -D --all-repeated=separate
If you pipe the output of the above into a text file, for example,
duplicates.txt
, you can then create a script from that:
awk '{$1="";printf("rm \"%s\"\n",$0);}' ~/duplicates.txt >~/duplicates.sh
Then edit the file and remove the lines for the files you want to keep, make the script executable and run it. Done.
Tags: cli, duplicates, find, awk, motd
Virt-Manager Pool Running Out of Space
Once upon a time, I had an issue with virt-manager’s pool space
running low. I was creating too many VMs using the default
configuration. This needed to be resolved because images are being
created in /var/lib/libvirt/images which was part of my
root
partition, which was also low. Since I am the only
user on this machine, I decided to change this to default to my home
directory:
- Create
~/libvirt/images
- Run
$ sudo virsh pool-edit default
as a privileged user or as theroot
user. - Change the path to point to your new directory.
Tags: cli, virt-manager, virsh, motd
KVM: Importing an OVA appliance
You may or may not be aware if it, but an OVA file is just a
tar archive containing an .ovf
and a .vmdk
files, respectively the VM configuration and disk.
$ ls *.ova
HTAOE.ova
$ tar tf HTAOE.ova
HTAOE.ovf
HTAOE-disk001.vmdk
HTAOE.mf
So, you can simply extract the files:
$ tar xvf HTAOE.ova
And convert to a format appropriate for QEMU/KVM:
List the available formats
$ qemu-img -h | tail -n4
Supported formats: blkdebug blklogwrites blkverify bochs cloop compress copy-before-write copy-on-read dmg file ftp ftps host_cdrom host_device http https luks nbd null-aio null-co nvme parallels preallocate qcow qcow2 qed quorum raw replication ssh throttle vdi vhdx vmdk vpc vvfat
See <https://qemu.org/contribute/report-a-bug> for how to report bugs.
More information on the QEMU project at <https://qemu.org>.
Do the actual conversion (I chose qcow2 here)
$ qemu-img convert -O qcow2 HTAOE.vmdk HTAOE.qcow2
Have a look at the .ovf
too, for information on expected
machine configuration, resources (eg. memory and cpu), etc.
After the conversion, simply create a new VM and make it use the newly created disk as the primary disk.
Tags: cli, kvm, ova, virtualization, qemu, motd
Compiling 32-bit C/C++ Programs on 64-bit
By default, gcc will compile your c/c++ program to 64-bit
instructions. In order to compile to 32-bit you need to add a
-m32
flag to the command line. For example, to compile a
file called hello-world.c using a Linux terminal, you would do
something like this:
$ gcc -m32 hello-world.c -o hello-world
If you get the following error:
fatal error: bits/predefs.h: No such file or directory
it means that the multilib standard library is missing. Therefore, you must install gcc-multlib. The name is different based upon the distribution and the compiler. For Debian/Ubuntu-based distros it would be:
$ sudo apt-get install gcc-multilib # For C
$ sudo apt-get install g++-multilib # For C++
Tags: cli, programming, c, cpp, multilib, motd
How to Clean Up the Fedora Root Folder
OK, so you’re running Fedora and you’re getting warnings that you’re running out of disk space on the root folder. What are your options?
The first thing to consider is resizing the root partition size. For that you can find instructions here
If that is not an option, then the next thing you probably should do
is find out which folders/directories are causing the issue. You can use
the du
command and some piping to figure this out:
Search the largest folders
$ sudo du --exclude="/home" -x -h -a / | sort -r -h | head -30
Common cleanable folders
Sometimes the issue is found in the docker cache:
$ docker system prune -a
or previous kernel versions:
$ sudo dnf remove $(dnf repoquery --installonly --latest-limit=-2 -q)
Journal logs
$ sudo journalctl --vacuum-size=100M
Caches
The dnf package cache can be cleaned up too:
$ sudo dnf clean packages
You can also try cleaning up the Fedora version cache:
$ sudo dnf system-upgrade clean
Tags: cli, fedora, disk-space, clean, resize, sysadmin, motd
Install Spotify using Flatpak using Command Line
If Flatpak is not installed, then install it:
$ sudo dnf install -y flatpak # Fedora
$ sudo apt install flatpak # Ubuntu-based
$ sudo zypper install flatpak # SUSE
$ sudo pacman -S flatpak # Arch-based
Since flathub.org is the most popular flatpak repository we will use the command line to enable it:
$ flatpak remote-add --if-not-exists flathub \
https://flathub.org/repo/flathub.flatpakrepo
Go to the Spotify page on Flathub and choose “Install”, or run the installation from the command line:
$ sudo flatpak install flathub com.spotify.Client
Tags: cli, flatpak, spotify, motd
How to reveal dot files in finder?
Here’s one from my archives. To reveal dot files in macOS Finder, open your Terminal (/Applications/Utilities) application and then enter,
defaults write com.apple.finder AppleShowAllFiles YES
Then restart Finder by holding down Option
and
Control
, or right-clicking on the Finder’s icon in your
Dock and choosing “Relaunch”.
To revert this behavior, simply change the YES
to
NO
in that defaults command.
Tags: cli, finder, macos, motd
Install Epson ET-3850 Printer Driver on Slackware 15.0
I recently bought an Epson ET-3850 printer from Costco to replace my HP OfficeJet which was about 10 years old and the family was starting to have issues with establishing a wi-fi connection to it on their Windoze laptops. It took quite a while to setup. The Epson seems to be doing the job. Time will tell, however I discovered that setting it up for Slackware would also take a bit of effort. Drivers are only available on the Epson website in RPM or DEB format so I had to create a Slackware package to install it. Here’s how I went about doing that.
- First, download the RPM package from the Epson Support website.
- Convert the RPM to a Slackware
.txz
package using therpm2txz
utility with the following options:
-c
: to make sure the file permissions are set correctly-S
: to extract the install scripts from the RPM (be careful with this)-n
: name the package using standard Slackware format-d
: make a slack-desc from the RPM’s metadata
# rpm2txz -c -S -n -d epson-inkjet-printer-escpr2-1.2.4-1.x86_64.rpm
- Now review the script. Make any changes, if necessary (I didn’t need to), and update the converted package:
# mkdir temp && cd temp
# explodepkg ../epson-inkjet-printer-escpr2-1.2.4-x86_64-1.txz
# makepkg -l y -c y ../epson-inkjet-printer-escpr2-1.2.4-x86_64-1.txz
- Once that is done you can install it:
# cd ..
# installpkg epson-inkjet-printer-escpr2-1.2.4-x86_64-1.txz
The printer did not show up as available to install in the “System Settings” app so go to the local CUPS webpage at http://localhost:631 and do the setup from there. Print a few test pages. It seems to be working fine on my Slackware laptop. You can move the package to your other Slackware machines and install there too.
Tags: cli, slackware, printers, epson, motd
Vim Keyboard Shortcuts
gg
Move to the first line of the fileG
Move to the last linegg=G
Reindent the whole filegv
Reselect the last visual selection<
Jump to beginning of last visual selection>
Jump to end of last visual selection^
Move to first non-blank character of the lineg_
Move the last non-blank character of the line (but you remove trailing whitespace, right)g_lD
Delete all the trailing whitespace on the lineea
Append to the end of the current wordgf
Jump to the file name under the cursorxp
Swap character forwardXp
Swap character backwardyyp
Duplicate the current lineyapP
Duplicate the current paragraphdat
Delete around an HTML tag, including the tagdit
Delete inside an HTML tag, excluding the tagw
Move one word to the rightb
Move one word to the leftdd
Delete the current linezc
Close current foldzo
Open current foldza
Toggle current foldzi
Toggle folding entirely<<
Outdent current line>>
Indent current linez=
Show spelling correctionszg
Add to spelling dictionaryzw
Remove from spelling dictionary~
Toggle case of current charactergUw
Uppercase until end of word (u for lower, ~ to toggle)gUiw
Uppercase entire word (u for lower, ~ to toggle)gUU
Uppercase entire linegu$
Lowercase until the end of the lineda"
Delete the next double-quoted string+
Move to the first non-whitespace character of the next lineS
Delete current line and go into insert modeI
insert at the beginning of the lineci"
Change what’s inside the next double-quoted stringca{
Change inside the curly braces (try [, (, etc.)vaw
Visually select worddap
Delete the whole paragraphr
Replace a character[
Jump to beginning of last yanked text]
Jump to end of last yanked textg;
Jump to the last change you madeg,
Jump back forward through the change list&
Repeat last substitution on current lineg&
Repeat last substitution on all linesZZ
Save the current file and quit Vim
Tags: cli, vim, cheatsheet, shortcuts, motd
Linkers
If you’re a developer who is interested in “how the sausage is made” then you might want to check out Ian Lance Taylor’s blog which contains a series of articles discussed on Klaatu’s GNU World Order Episode 400 about the linker
Here are the links to the articles:
Linkers part 1 Linkers part 2 Linkers part 3 Linkers part 4 Linkers part 5 Linkers part 6
Tags: cli, linkers, programming, compiling, motd
Mounting a Remote Drive using FUSE
If you have FUSE (Filessystem in Userspace) installed, you
can use SSH to mount a remote drive locally. This is very
simple to do and you don’t need to be root. You just need
SSH access to the remote system. Just create a directory to
mount the system and use the sshfs
command to connect the
remote to it.
Here I will mount my home directory on my remote server named slacker locally to a directory on my laptop’s home directory:
$ cd ~
$ mkdir slacker
$ sshfs user@192.168.1.99:/home/user/ slacker
$ cd slacker/
$ ls
Backups KVM Public bin iso public_html
Calibre\ Library Music PycharmProjects cv_debug.log logs tmp
Desktop OneDrive README.txt file.txt lynis-report.dat
Documents Pictures Templates fortune.txt lynis.log
Downloads Podcasts Videos go pCloudDrive
The above is a listing of my home directory on the remote.
Install Joplin on Slackware 15.0
On a recent episode of Hacker Public Radio, the host, Lee listed several note-taking apps that he had tried to use. One of them was Joplin which I have been using for several years now. I feel that it is probably one of the best note-taking apps available that:
- Supports Markdown
- Is cross-platform
- Supports vim keyboard mappings (I can’t live without this)
- Integrates with multiple cloud storage environments.
I use it on a daily basis for almost all of my notes but it also supports checklists on my iPhone, iPad, Windoze laptop, and all of my Slackware devices (desktop and laptops). Give it a try!
As of yet, it is not available as a SlackBuild, but you can install it on Slackware quite easily using the CLI with:
$ wget -O - https://raw.githubusercontent.com/laurent22/joplin/dev/Joplin_install_and_update.sh | bash
Have fun!
Tags: cli, joplin, slackware, note-apps, motd
SlackBuilds: Where Are They Located?
You should find a file in /var/log/packages
for each
SlackBuild that you have installed. You can distinguish the
SlackBuilds package from standard Slackware packages by the
substring SBo embedded in the filename. So you can create a
list of all of the SlackBuilds installed on your machine by
just doing an ls *SBo*
in that directory.
ls /var/log/packages | grep SBo
or
ls /var/log/packages/*SBo
If you plan on reinstalling it may be a good idea to pipe that information out to a text file.
ls /var/log/packages | grep SBo > SBo_packages.txt
If you cat
one of those files you will display a text
file that lists certain information about the package, the location of
the package and a brief description. For example, the file for
sigil package starts like this:
PACKAGE NAME: sigil-0.9.12-x86_64-1_SBo
COMPRESSED PACKAGE SIZE: 7.7M
UNCOMPRESSED PACKAGE SIZE: 25M
PACKAGE LOCATION: /tmp/sigil-0.9.12-x86_64-1_SBo.tgz
PACKAGE DESCRIPTION:
sigil: sigil (Multi-platform WYSIWYG ebook editor)
sigil:
sigil: Sigil is a multi-platform WYSIWYG ebook editor. It is designed to
sigil: edit books in ePub format.
sigil:
sigil: Homepage: https://sigil-ebook.com/
sigil:
It then goes on and displays a file list of all of the files contained by the package and where they were installed.
Armed with the above information, you can the cd
to the
/tmp
directory and find the package itself:
$ cd /tmp
$ ls -l sigil-0.9.12-x86_64-1_SBo.tgz
-rw-r--r-- 1 root root 8070041 May 29 16:24 sigil-0.9.12-x86_64-1_SBo.tgz
$
Tags: cli, slackware, slackbuilds, motd
Slackware Kernel Upgrade (elilo)
If you are using elilo
with Slackware, after running the
slackpkg --upgrade-all
script and you see that a new kernel
was installed, you need to run the following commands to move the latest
kernel to its proper location in order to boot:
geninitrd
cp /boot/vmlinuz-generic /boot/efi/EFI/Slackware/vmlinuz
cp /boot/initrd.gz /boot/efi/EFI/Slackware/initrd.gz
How to Change the UID or GID for a User or Group
Let’s assume we have a user, Jane Doe, with the username of jdoe and the UID of 1001 who we need to move to another UID (for some reason or another).
First, change the UID of the user:
# usermod -u 3001 jdoe
Next, use search and change all file’s ownership owned by this user:
# find / -user 1001 -exec chown -h jdoe {} \;
What if we need to change the GID of a group? Basically the same process can be used. Let’s say we want to change the admins group’s GID from 2001 to 4001.
First, change the GID:
# groupmod -g 4001 admins
Now, use search and change all file’s ownership owned by this group:
# find / -group 2001 -exec chgrp -h admins {} \;
Tags: cli, sysadmin, uid, gid, motd
Netstat
My most frequently used netstat
command with parameters
for checking port (active internet) connections is:
$ netstat -tulpn
You can also pipe the output to grep
to filter for
specific ports or addresses.
Here is a brief listing of some of the many options:
$ netstat [options]
Option | Action |
---|---|
-a | Display the state of all sockets, not just the active. |
-c | Continuously display information, updating every second. |
-i | Include network device statistics. |
-n | Display all network addresses as numbers. |
-o | Display additional information. |
-r | Display routing tables. |
-t | Display only the TCP sockets. |
-u | Display only the UDP sockets. |
-v | Print the netstat version number and exit. |
-w | List only the raw sockets. |
-x | Display only the Unix sockets. |
Tags: cli, networking, netstat, motd
Extracting Client Certificates
You can display all of the server’s certificates using the following command:
$ openssl s_client -showcerts server.name:port | sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p'
This output can be piped or copy/pasted to a text file (only keep the
parts between the BEGIN and END CERTIFICATE sections)
and give it a .crt
file extension. You can then use it as
input to whatever client app needs it.
In the below example we pipe the openssl
output to
grep
to remove the identifiers of the certificates. You may
not get your prompt back until the command times out, so you
should wait a bit and after a bit you should get something similar to
the following output:
$ openssl s_client -showcerts example.com:443 2>&1 |grep -v '^depth'| sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p'
-----BEGIN CERTIFICATE-----
MIIHSjCCBjKgAwIBAgIQDB/LGEUYx+OGZ0EjbWtz8TANBgkqhkiG9w0BAQsFADBP
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMSkwJwYDVQQDEyBE
aWdpQ2VydCBUTFMgUlNBIFNIQTI1NiAyMDIwIENBMTAeFw0yMzAxMTMwMDAwMDBa
Fw0yNDAyMTMyMzU5NTlaMIGWMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZv
cm5pYTEUMBIGA1UEBxMLTG9zIEFuZ2VsZXMxQjBABgNVBAoMOUludGVybmV0wqBD
b3Jwb3JhdGlvbsKgZm9ywqBBc3NpZ25lZMKgTmFtZXPCoGFuZMKgTnVtYmVyczEY
MBYGA1UEAxMPd3d3LmV4YW1wbGUub3JnMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A
MIIBCgKCAQEAwoB3iVm4RW+6StkR+nutx1fQevu2+t0Fu6KBcbvhfyHSXy7w0nJO
dTT4jWLjStpRkNQBPZwMwHH35i+21gdnJtDe/xfO8IX9McFmyodlBUcqX8CruIzD
v9AXf2OjXPBG+4aq+03XKl5/muATl32++301Vw1dXoGYNeoWQqLTsHT3WS3tOOf+
ehuzNuZ+rj+ephaD3lMBToEArrtC9R91KTTN6YSAOK48NxTA8CfOMFK5itxfIqB5
+E9OSQTidXyqLyoeA+xxTKMqYfxvypEek1oueAhY9u67NCBdmuavxtfyvwp7+o6S
d+NsewxAhmRKFexw13KOYzDhC+9aMJcuJQIDAQABo4ID2DCCA9QwHwYDVR0jBBgw
FoAUt2ui6qiqhIx56rTaD5iyxZV2ufQwHQYDVR0OBBYEFLCTP+gXgv1ssrYXh8vj
gP6CmwGeMIGBBgNVHREEejB4gg93d3cuZXhhbXBsZS5vcmeCC2V4YW1wbGUubmV0
ggtleGFtcGxlLmVkdYILZXhhbXBsZS5jb22CC2V4YW1wbGUub3Jngg93d3cuZXhh
bXBsZS5jb22CD3d3dy5leGFtcGxlLmVkdYIPd3d3LmV4YW1wbGUubmV0MA4GA1Ud
DwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwgY8GA1Ud
HwSBhzCBhDBAoD6gPIY6aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0
VExTUlNBU0hBMjU2MjAyMENBMS00LmNybDBAoD6gPIY6aHR0cDovL2NybDQuZGln
aWNlcnQuY29tL0RpZ2lDZXJ0VExTUlNBU0hBMjU2MjAyMENBMS00LmNybDA+BgNV
HSAENzA1MDMGBmeBDAECAjApMCcGCCsGAQUFBwIBFhtodHRwOi8vd3d3LmRpZ2lj
ZXJ0LmNvbS9DUFMwfwYIKwYBBQUHAQEEczBxMCQGCCsGAQUFBzABhhhodHRwOi8v
b2NzcC5kaWdpY2VydC5jb20wSQYIKwYBBQUHMAKGPWh0dHA6Ly9jYWNlcnRzLmRp
Z2ljZXJ0LmNvbS9EaWdpQ2VydFRMU1JTQVNIQTI1NjIwMjBDQTEtMS5jcnQwCQYD
VR0TBAIwADCCAX8GCisGAQQB1nkCBAIEggFvBIIBawFpAHYA7s3QZNXbGs7FXLed
tM0TojKHRny87N7DUUhZRnEftZsAAAGFq0gFIwAABAMARzBFAiEAqt+fK6jFdGA6
tv0EWt9rax0WYBV4re9jgZgq0zi42QUCIEBh1yKpPvgX1BreE0wBUmriOVUhJS77
KgF193fT2877AHcAc9meiRtMlnigIH1HneayxhzQUV5xGSqMa4AQesF3crUAAAGF
q0gFnwAABAMASDBGAiEA12SUFK5rgLqRzvgcr7ZzV4nl+Zt9lloAzRLfPc7vSPAC
IQCXPbwScx1rE+BjFawZlVjLj/1PsM0KQQcsfHDZJUTLwAB2AEiw42vapkc0D+Vq
AvqdMOscUgHLVt0sgdm7v6s52IRzAAABhatIBV4AAAQDAEcwRQIhAN5bhHthoyWM
J3CQB/1iYFEhMgUVkFhHDM/nlE9ThCwhAiAPvPJXyp7a2kzwJX3P7fqH5Xko3rPh
CzRoXYd6W+QkCjANBgkqhkiG9w0BAQsFAAOCAQEAWeRK2KmCuppK8WMMbXYmdbM8
dL7F9z2nkZL4zwYtWBDt87jW/Gz/E5YyzU/phySFC3SiwvYP9afYfXaKrunJWCtu
AG+5zSTuxELFTBaFnTRhOSO/xo6VyYSpsuVBD0R415W5z9l0v1hP5xb/fEAwxGxO
Ik3Lg2c6k78rxcWcGvJDoSU7hPb3U26oha7eFHSRMAYN8gfUxAi6Q2TF4j/arMVB
r6Q36EJ2dPcTu0p9NlmBm8dE34lzuTNC6GDCTWFdEloQ9u//M4kUUOjWn8a5XCs1
263t3Ta2JfKViqxpP5r+GvgVKG3qGFrC0mIYr0B4tfpeCY9T+cz4I6GDMSP0xg==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIEvjCCA6agAwIBAgIQBtjZBNVYQ0b2ii+nVCJ+xDANBgkqhkiG9w0BAQsFADBh
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBD
QTAeFw0yMTA0MTQwMDAwMDBaFw0zMTA0MTMyMzU5NTlaME8xCzAJBgNVBAYTAlVT
MRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMxKTAnBgNVBAMTIERpZ2lDZXJ0IFRMUyBS
U0EgU0hBMjU2IDIwMjAgQ0ExMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC
AQEAwUuzZUdwvN1PWNvsnO3DZuUfMRNUrUpmRh8sCuxkB+Uu3Ny5CiDt3+PE0J6a
qXodgojlEVbbHp9YwlHnLDQNLtKS4VbL8Xlfs7uHyiUDe5pSQWYQYE9XE0nw6Ddn
g9/n00tnTCJRpt8OmRDtV1F0JuJ9x8piLhMbfyOIJVNvwTRYAIuE//i+p1hJInuW
raKImxW8oHzf6VGo1bDtN+I2tIJLYrVJmuzHZ9bjPvXj1hJeRPG/cUJ9WIQDgLGB
Afr5yjK7tI4nhyfFK3TUqNaX3sNk+crOU6JWvHgXjkkDKa77SU+kFbnO8lwZV21r
eacroicgE7XQPUDTITAHk+qZ9QIDAQABo4IBgjCCAX4wEgYDVR0TAQH/BAgwBgEB
/wIBADAdBgNVHQ4EFgQUt2ui6qiqhIx56rTaD5iyxZV2ufQwHwYDVR0jBBgwFoAU
A95QNVbRTLtm8KPiGxvDl7I90VUwDgYDVR0PAQH/BAQDAgGGMB0GA1UdJQQWMBQG
CCsGAQUFBwMBBggrBgEFBQcDAjB2BggrBgEFBQcBAQRqMGgwJAYIKwYBBQUHMAGG
GGh0dHA6Ly9vY3NwLmRpZ2ljZXJ0LmNvbTBABggrBgEFBQcwAoY0aHR0cDovL2Nh
Y2VydHMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0R2xvYmFsUm9vdENBLmNydDBCBgNV
HR8EOzA5MDegNaAzhjFodHRwOi8vY3JsMy5kaWdpY2VydC5jb20vRGlnaUNlcnRH
bG9iYWxSb290Q0EuY3JsMD0GA1UdIAQ2MDQwCwYJYIZIAYb9bAIBMAcGBWeBDAEB
MAgGBmeBDAECATAIBgZngQwBAgIwCAYGZ4EMAQIDMA0GCSqGSIb3DQEBCwUAA4IB
AQCAMs5eC91uWg0Kr+HWhMvAjvqFcO3aXbMM9yt1QP6FCvrzMXi3cEsaiVi6gL3z
ax3pfs8LulicWdSQ0/1s/dCYbbdxglvPbQtaCdB73sRD2Cqk3p5BJl+7j5nL3a7h
qG+fh/50tx8bIKuxT8b1Z11dmzzp/2n3YWzW2fP9NsarA4h20ksudYbj/NhVfSbC
EXffPgK2fPOre3qGNm+499iTcc+G33Mw+nur7SpZyEKEOxEXGlLzyQ4UfaJbcme6
ce1XR2bFuAJKZTRei9AqPCCcUZlM51Ke92sRKw2Sfh3oius2FkOH6ipjv3U/697E
A7sKPPcw7+uvTPyLNhBzPvOk
-----END CERTIFICATE-----
Tags: cli, certificates, openssl, motd
Use the zdump Command to Check Daylight Savings Time Settings
If you are one of us lucky folks who live in timezones that
observe Daylight Savings Time, you can use the
zdump
timezone dumper command to view your server’s
settings for same. This command prints the current time in each timezone
provided on the command line.
You can use grep
to reduce the output to just the lines
that reference the current year:
$ zdump -v EST5EDT |grep '2023'
EST5EDT Sun Mar 12 06:59:59 2023 UT = Sun Mar 12 01:59:59 2023 EST isdst=0 gmtoff=-18000
EST5EDT Sun Mar 12 07:00:00 2023 UT = Sun Mar 12 03:00:00 2023 EDT isdst=1 gmtoff=-14400
EST5EDT Sun Nov 5 05:59:59 2023 UT = Sun Nov 5 01:59:59 2023 EDT isdst=1 gmtoff=-14400
EST5EDT Sun Nov 5 06:00:00 2023 UT = Sun Nov 5 01:00:00 2023 EST isdst=0 gmtoff=-18000
You can also run the same command by location:
$ zdump -v /usr/share/zoneinfo/America/New_York|grep '2023'
/usr/share/zoneinfo/America/New_York Sun Mar 12 06:59:59 2023 UT = Sun Mar 12 01:59:59 2023 EST isdst=0 gmtoff=-18000
/usr/share/zoneinfo/America/New_York Sun Mar 12 07:00:00 2023 UT = Sun Mar 12 03:00:00 2023 EDT isdst=1 gmtoff=-14400
/usr/share/zoneinfo/America/New_York Sun Nov 5 05:59:59 2023 UT = Sun Nov 5 01:59:59 2023 EDT isdst=1 gmtoff=-14400
/usr/share/zoneinfo/America/New_York Sun Nov 5 06:00:00 2023 UT = Sun Nov 5 01:00:00 2023 EST isdst=0 gmtoff=-18000
Tags: cli, zdump, timezones, motd
Logging to the System Logger in Bash
If you are writing a shell script and want any messages to appear in
the system logger you can just use the logger
command to do
this.
Here is an example on my Slackware system:
$ logger -t TEST "This is a test message"
$ cat /var/log/messages
cat: /var/log/messages: Permission denied
$ su -
Password:
# cat /var/log/messages
Aug 23 17:44:47 slackbook TEST: This is a test message
You can also log to systemd by providing a parameter to logger:
logger --journald <<end
MESSAGE_ID=67feb6ffbaf24c5cbec13c008dd72309
MESSAGE=The dogs bark, but the caravan goes on.
DOGS=bark
CARAVAN=goes on
end
or you can optionally provide a file whose contents will be logged:
logger --journald=entry.txt
See the logger manpage for additional information.
Tags: cli, logging, systemd, logger, motd
Encrypting Files using OpenSSL
Let’s say we have a file that contains sensitive information and we
want to encyrpt it. You can encrypt a file very easily using the
openssl
command:
$ cat secret.txt
This file contains some very secret stuff
$ openssl enc -e -aes-256-cbc -pbkdf2 -a -salt -in secret.txt -out secret.enc
enter aes-256-cbc encryption password: <enter-a-password>
Verifying - enter aes-256-cbc encryption password: <enter-a-password>
$ cat secret.enc
U2FsdGVkX19Rnz48WjLeljd19wvNOhQy+zzYwxCANezCTkqpGMl9zs4HdwdUzZjl
VQkUsCJ7b0rUpRi83UlcwA==
Now we decrypt it (output to a different file). Make sure you enter the same password that you input above:
$ openssl enc -d -aes-256-cbc -pbkdf2 -a -salt -in secret.enc -out unencrypted-secret.txt
enter aes-256-cbc decryption password: <enter-a-password>
$ cat unencrypted-secret.txt
This file contains some very secret stuff
$
Tags: cli, openssl, encryption, motd
Finding the SSL Directory on a Server
I’ve had situations where I was configuring a secure connection to an
application and needed to know where the SSL certificates are stored on
the server. You can easily find out this information using the
openssl
and grep
commands:
$ openssl version -a | grep OPENSSLDIR
OPENSSLDIR: "/etc/pki/tls"
Processing The Results of The Find Command
As mentioned in the previous post, the find command searches for files, but it has another very useful feature in that it can also perform an action on the files it finds.
I am a vim user, and let’s assume that I want to find my editor’s backup files in the current directory trees. These filesall end with a tilda (~) character. We would use this command to search for them:
$ find . -name '*~'
./.buffer.un~
./.find_buffer.txt.un~
./.Tips.txt.un~
which results in a list of 3 files. All it takes to remove these files is to add on a little to the end of the last command:
$ find . -name '*~' -exec rm -i '{}' \;
Now, not only will this command find all the matching files in the current directory tree, but it will also delete them.
The -exec
parameter tells find
to
execute the command that follows it on each result. The string
{}
is replaced by the current file name being processed
everywhere it occurs in the arguments to the command. The
{}
string is enclosed in single quotes to protect them from
interpretation as shell script punctuation. The \;
sequence
indicates the end of the -exec
argument.
See the manpage for more information.
Using the Find Command to Search for Files
One of the most useful and flexible GNU utilities is the find command. Understanding how to use this command is very important to make you Linux life more efficient.
The general syntax of the find command is:
find [-H] [-L] [-P] [-D debugopts] [-Olevel] [starting-point...] [expression]
That looks like a lot, but most of the time you may only need 2 things:
find [path] [expression]
where the path is a starting point or the top of a directory tree to be searched, and expression is a property and value pair of what you’re trying to find. This may be a file name, last access time, last modification time, size, and/or ownership.
For example, if you’re looking for the file stdlib.h, use the following command:
find / -name stdlib.h
If you run this as a normal user, using find from
the root directory will often result in a lot of error messages
being output to the terminal because the normal user doesn’t
have access to view some of the directories in the search. Therefore you
may want to pipe the stderr output to /dev/null
to
avoid seeing those messages. You can do that like this:
find / -name stdlib.h 2>/dev/null
Git Aliases
Once you start developing some experience with using git on the command line, you will sometimes find yourself typing some long commands over and over again. You can create aliases in git that are very similar to the bash alias shortcuts you use in the shell.
Steps to do this are highlighted in chapter 2.7 of the Git Book authored by Scott Chacon and Ben Straub and published by Apress.
One of the commands that I use a lot is:
$ git log --graph --decorate --pretty=oneline --abbrev-commit --all --date=local
which provides a nice colorized listing of the project’s activity and looks something like this:
* 2f6cc07 (HEAD -> automation, origin/master, origin/HEAD, master) Updated merge for README.md
|\
| * 84e20c2 Initial commit
* 8dc456f Completed migration for github site.
* bf67601 fixed head before sed
* 89780c3 Merge pull request #124 from thalios1973/feat/pandoc-test-fix
|\
| * c9a297c Added additional check that will allow the use of pandoc without the --strict flag or 'hsmarkdown' hack.
* | 025cf79 Merge pull request #125 from thalios1973/feat/body-end-file
|\ \
| |/
|/|
| * 272b1b6 Added ability to include custom code just before the </body> tag. body_end_file global config variable added.
|/
* 9f66ad0 README formatting
* 500253e Support for static, not managed by bashblog html files. Close #92
* 3c73ef6 Deleted the now defunct Twitter JSON API for share count. Fix #117
* b5a4590 bump version to 2.8
* 5c8e0e5 Revert changes in #116
* 5fc037f Merge branch 'master' of github.com:cfenollosa/bashblog
|\
| * c6a9bef Revert tag management from #116
| * 6222672 Better error message for $EDITOR. Close #112
| * 36d79b5 support Markdown.pl in bashblog folder. Close #113
| * 7154c07 Merge pull request #116 from McDutchie/master
| |\
| | * f50a17c tags_in_post(): bugfix for non-GNU 'sed'
| | * 2a29b22 Fix renaming using 'bb.sh edit -n'. Suppress 'which' errmsg.
| | * 62a26bb Merge remote-tracking branch 'upstream/master' Resolve minor editing 2.6-to-2.7 editing conflict in bb.sh
| | |\
| | * | 54cc0c8 More code refactoring. Limit word splitting and disable globbing by default.
| | * | d1a84d6 Merge remote-tracking branch 'upstream/master'
| | |\ \
| | * | | a674ec5 rebuild_tags(): use array for more robust file handling
| * | | | 2157b92 Slavic language support, thanks to Tomasz Jadowski
| | |_|/
| |/| |
but nicer than this website can depict (for now).
Rather than typing this long command every time, I just create an alias:
$ git config --global alias.lola 'log --graph --decorate --pretty=oneline --abbrev-commit --all --date=local'
Then all I need to type from now on is:
$ git lola
Setting Up Git to Ignore ELF Binaries (C/C++ Output on Linux)
The starting point for this experiment was from here
An Example using C Program Source
Let’s say we have a pre-existing directory of some C source code files:
$ ls
Makefile print print.c print.h print.o
And we initialize a new git repository in that directory:
$ git init
Initialized empty Git repository in /home/user/tmp/printer/.git/
$ cat .gitignore
# Ignore all
*
# Unignore all with extensions
!*.*
# Unignore all dirs
!*/
# Unignore make files
!Makefile
# Ignore .o files
*.o
# Ignore
bindir
bin/
# or
*/bin/*
Let’s see how we did:
$ git status
On branch master
No commits yet
Untracked files:
(use "git add <file>..." to include in what will be committed)
.gitignore
Makefile
print.c
print.h
nothing added to commit but untracked files present (use "git add" to track)
So the print
and the print.o
files are not
showing up, which was our initial goal.
You may have to tweak the settings in the above
.gitignore
file to your own situation, but as you can see
it is possible to setup git to ignore the ELF binaries output
by gcc. I would probably also add a.out
to the
list of unignored files just to cover those times when you’re
not using a Makefile
. YMMV.
Processing a List of Files from an Input File
Suppose we have a list of files stored in a text file and we want to perform an operation on each of them within a bash script. How would we go about doing that? Well, there are several options, here are a few.
$ cat input-files.txt
file-1.txt
file-2.txt
file-3.txt
file-4.txt
file-5.txt
file-6.txt
file-7.txt
file-8.txt
file-9.txt
Now, if we want to operate on each file in this list we can do something like this:
while IFS= read -r filename
do
echo "Do something on ${filename} here..."
done < "input-files.txt"
or alternatively,
input="input-files.txt"
while IFS= read -r filename
do
printf '%s\n' "${filename}"
done < "${input}"
Tags: cli, bash, shell-scripting, motd
String Processing with Bash
There are various tools built into bash that enable you to manipulate a variable or string which come in handy when writing shell scripts. Here are a few notable ones:
Find the length of a string
${#string}
Get a Substring from a String
${string:pos} or ${string:pos:len}
Removing Substrings from a String
${string#substr}
${string%substr}
${string##substr}
${string%%substr}
Some examples
Here’s a few examples of how you can process a variable that points to an absolute path of a file and shows how to extract certain parts of said file’s path:
var=/home/user/code/blogstuff/index.html.j2
echo ${var} # => /home/user/code/blogstuff/index.html.j2
echo ${var#*.} # => html.j2
echo ${var##*.} # => j2
echo ${var%/*.*} # => /home/user/code/blogstuff
Replace a Substring of a String
${string/pattern/substr} # => Replaces the 1st match found
${string//pattern/substr} # => Replaces all of the matches found
Replace the Beginning or End of a String
${string/#pattern/substr}
${string/%pattern/substr}
file=${var##/*/} # => index.html.j2
echo ${file/#index/fubar} # => fubar.html.j2
echo ${file/%j2/fubar} # => index.html.fubar
Tags: cli, bash, shell-scripting, motd
Convert a Git Repo to a Bare Repo
If you have a git repository that you haven’t cloned from a remote location, i.e., one that was created locally, it is easy to convert it to a bare repository. This process describes how to:
- Take a “normal” git repository
- Move the .git directory to another location
- Convert it to a bare repository
Suppose you have a git repository called repo. To convert it to a bare repository, execute the following commands:
cd repo
mv .git ../repo.git # renaming just for clarity
cd ..
rm -fr repo
cd repo.git
git config --bool core.bare true
Now you can clone it like a “normal” repository:
$ cd ..
$ git clone repo.git myrepo
$ cd myrepo/
$ ls
file-1.txt file-2.txt file-3.txt file-4.txt file-5.txt
$ git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean
$
Please note that repositories from a remote location do not have all of their git branch contents and history stored locally, so doing this to a repository cloned from a remote will result in a repository that will be missing data.
Git: Working with Git for the First Time
Here’s the first step every user should start with after downloading and installing git but BEFORE doing anything else: Set up your global environment variables! If you don’t do this git will abort your commit and give you an annoying message like this one:
$ git commit file.txt
Author identity unknown
*** Please tell me who you are.
Run
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
to set your account's default identity.
Omit --global to set the identity only in this repository.
fatal: empty ident name (for <user@localhost>) not allowed
You only should have to do the git config --global...
commands one time. If you want to use a different user ID and email
address in each repository then don’t include the --global
parameter and it will only update your current repository leaving your
global setting alone. If you’re contributing to different repositories
using different email addresses or IDs then you definitely want to get
into this habit.
Git: Committing to a Different Branch
Let’s imaging that you’ve made some changes to files in your work directory and then realized that you aren’t in the correct branch. Now you want to get the changes you’ve made to the right branch but no change the current branch. What do you do?
There is no git command to do this but you can use the git stash command to fix this.
This involves:
- Using the git stash command to temporarily store our changes elsewhere,
- Checkout the correct branch, and
- “Unstash” the changes to the correct branch.
Let’s say that we’ve made some changes in our repository’s new-release branch but they should have been made in the new-feature branch.
git stash
git checkout new-feature
git stash pop
We can now perform a commit, and update this branch.
git commit -a -m "Updated functions in new-feature branch."
Printing from Virtualbox
If you want to set up your system so that you can access your Linux host’s printer from a Windows VirtualBox client then:
On Your Linux Host
- Install your printer driver. Get your printer working on your host first.
- Install CUPS if it is not already installed.
- Add your printer to CUPS at
http://localhost:631/admin
. - Set your virtual machine’s Network to Bridged Adapter
(eth0) in Settings and connect the cable as necessary (Advanced
menu).
- Startup VirtualBox.
- In a terminal, enter
ifconfig
to list your current network settings and copy down your host’s (eth0) address or you can optionally access this address using Network Manager.
On You Windows Client
- Add a new Network printer in “Printers and Faxes” from the Control Panel
- Enter the URL address of your host’s printer, i.e.,
http://192.168.0.2:631/printers/Your_printer's_name
. You need the:631
after your host’s address as this is the port that CUPS listens to. The Your_printer’s_name is the name of the printer you added to CUPS.
If you can’t establish a connection then check your firewall settings.
Tags: cli, virtualbox, printing, motd
Installing and Using sbotools in Slackware 15.0
Slackware was the first Linux distribution that I ever used but I was always curious and did a lot of distro-hopping from Slackware, Caldera, Turbo Linux, Mandrake, Red Hat, Ubuntu, multiple Ubuntu derivatives, then Fedora for quite a while until I finally settled on…(drum roll please)…Slackware! Yeah, I’ve come full circle.
One of the reasons why I was doing so much hopping around is that I was never really happy with something about the distribution that I was currently using. Maybe it was the default colors (Ubuntu, come on with all of that orange stuff), choice of default software, difficulty with setting it up. Whatever. With Slackware it was always the (lack of) a decent package management system. That is until I found “sbotools”.
Yeah, slackpkg is there. So are a few others, like (list a few of them here). But the lack of a distribution that didn’t have dependency management in its package manager is a real pain in the backside.
So what’s the big deal? Well, one of the points of running Slackware is installing packages that you choose to install and not letting the package manager run amuck and automatically select and install a full list of dependencies without letting the user know what they are and what they do. That’s where sbotools comes in.
Klaatu talks about sbotools in his Gnu World Order Episode 473 so you can find more information about it in his review.
As sbotools is highly influenced by FreeBSD’s ports system and its pkgtools, the interface will be more or less familiar to anyone familiar with portupgrade, etc.
When you use sbotools it finds the dependencies for the software you want to install and for each one, displays what it is, what it does, and prompts you if you want to install it or not. Also, if the dependency has optional parameters it asks you if you want to see them and gives you a choice to modify them or not.
Now all of this is done in the console, which is another thing that I like about sbotools. No fancy ncurses or GUI dialogs. Just a simple console program which is handy if you’re using a headless server.
Once you are done selecting your package and providing whatever responses are required, sbotools downloads the Slackbuilds and the source code from wherever it needs to and compiles the packages one by one in the order required and installs them. If a compilation fails then it asks you if you wish to continue or quit. Then you have the option to try figuring out what went wrong but looking at the build logs or by scrolling back on the console output.
The sbotools package is not a single program but a collection of about a half a dozen programs that work very much like the BSD ports. You have a tool to initialize and fetch the latest list of packages, another to check to see if you need to update your locally installed programs, another to search for them and one to install new packages.
Installing sbotools is rather simple if you’re use to installing packages by hand in Slackware. It comes as a tz package that you install using the ‘installpkg’ command.
First you need to install the sbotools package. Then you need to “sync” the SlackBuild repository to your machine using the command:
# sbosnap fetch
This will create a git repository of all of the SlackBuilds supported by this tool (which is a lot). Now you can use the sbofind command to search for a package.
Next we will run sbocheck to see if there are any updates since the last time. In this case since this is the first time we’re using it we shouldn’t expect anything new. The example below shows the results when you do have new results:
# sbocheck
Updating SlackBuilds tree...
Updating files: 100% (44404/44404), done.
HEAD is now at 49808a413d 20230729.1 global branch merge.
Checking for updated SlackBuilds...
plexmediaserver 1.32.5.7328_2632c9d3a < needs updating (1.32.5.7349_8f4248874 from SBo)
A copy of the above result is kept in /var/log/sbocheck.log
If the above command indicates that there are updates then you can initiate an upgrade using the following command:
# sboupgrade --all
Checking for updated SlackBuilds...
OK, so now your repository is set up. Let’s use it.
Suppose we want to find a software package that can read guitar tabs:
# sbofind guitar
SBo: tuxguitar 1.5.1
Path: /usr/sbo/repo/audio/tuxguitar
SBo: guitarix 0.44.1
Path: /usr/sbo/repo/audio/guitarix
We received two results. Let’s query their details:
# sbofind -r guitarix
SBo: guitarix 0.44.1
Path: /usr/sbo/repo/audio/guitarix
README:
guitarix (virtual guitar amplifier for jack)
guitarix offers the range of sounds you would expect from a
full-featured universal guitar-amp. You can get crisp clean-sounds,
nice overdrive, fat distortion and a diversity of crazy sounds never
heard before. Guitarix can be used as a standalone application, as
LADSPA plugins, or as LV2 plugins.
For 32-bit x86, a CPU with SSE instructions is required. For other
architectures, SSE will be used if available.
meterbridge is an optional runtime dependency.
Starting with version 0.35.4, guitarix's 'Online presets' feature no
longer requires webkitgtk. See README.online for more information.
This package uses POSIX filesystem capabilities to execute with
elevated privileges (required for realtime audio processing). This
may be considered a security/stability risk. Please read
http://www.slackbuilds.org/caps/ for more information. To disable
capabilities, pass SETCAP=no to the script.
root@slacker:~# sbofind -r tuxguitar
SBo: tuxguitar 1.5.1
Path: /usr/sbo/repo/audio/tuxguitar
README:
TuxGuitar is a multitrack guitar tablature editor and player written
in Java-SWT. It can open GuitarPro, PowerTab, and TablEdit files.
Ah, looks like tuxguitar is the one we want. So let’s install it:
# sboinstall tuxguitar
This is a SlackBuild to repackage the Azul's Zulu build of OpenJDK that
is compliant with the Java SE 8 standard.
Before installing this package please consider any other jdk's/jre's
that you have already installed as they may cause conflicts with the
PATH, JAVA_HOME and MANPATH variables.
After installing the package you will need to logout/reboot your machine
as it will add files to the /etc/profile.d folder.
Note: zulu-openjdk8 does not have a browser plugin nor support for Java
Network Launching Protocol (JNLP).
Proceed with zulu-openjdk8? [y] n
I already have a different version of Java installed so I will skip this part
TuxGuitar is a multitrack guitar tablature editor and player written
in Java-SWT. It can open GuitarPro, PowerTab, and TablEdit files.
Proceed with tuxguitar? [y]
tuxguitar added to install queue.
Install queue: tuxguitar
Are you sure you wish to continue? [y]
Slackware package /tmp/tuxguitar-1.5.1-x86_64-1_SBo.tgz created.
+==============================================================================
| Installing new package /tmp/tuxguitar-1.5.1-x86_64-1_SBo.tgz
+==============================================================================
Verifying package tuxguitar-1.5.1-x86_64-1_SBo.tgz.
Installing package tuxguitar-1.5.1-x86_64-1_SBo.tgz:
PACKAGE DESCRIPTION:
# tuxguitar (A Multitrack tablature editor and player)
#
# TuxGuitar is a multitrack guitar tablature editor and player
# written in Java-SWT. It can open GuitarPro, PowerTab, and
# TablEdit files.
#
Executing install script for tuxguitar-1.5.1-x86_64-1_SBo.tgz.
Package tuxguitar-1.5.1-x86_64-1_SBo.tgz installed.
Cleaning for tuxguitar-1.5.1...
#
So, sboinstall will not install any package without telling you first and it will describe what the packages does and if there are any additional (optional or otherwise) command line parameters which it will prompt you for.
Once you respond to the last prompt it will download and install (and compile if necessary) all of the packages you requested. If it fails one of the installations it will ask you if it should continue. If it succeeds in installing all of the packages then you’re ready to start using your software.
Each of the command line tools has it’s own manpage. Have fun!
Tags: cli, slackware, sbotools, slackbuilds, motd
Sorting Stuff in the VIM Buffer
Suppose I have a file that I am editing in vim with the following contents:
$ cat file.txt
Red 4
Blue 3
Green 1
Orange 7
Black 8
Yellow 6
Purple 2
White 5
If I want to sort this data in vim, the first thing to do is to format it into columns using the column command:
:%!column -t
Which will put the data in the following format:
Red 4 Blue 3 Green 1 Orange 7 Black 8 Yellow 6 Purple 2 White 5
Then we can sort on the 2nd column using the sort command:
:%!sort -k2nr
Which will update the buffer like this:
Black 8 Orange 7 Yellow 6 White 5 Red 4 Blue 3 Purple 2 Green 1
Using crontab
Crontab is a utility found on Linux and UNIX that allows you to schedule repetitive tasks on your system, like backups, updates, etc.
The crontab command manages a table containing specially formatted lines of text which is processed periodically by the system. Each line consists of a “schedule” followed by one or more commands to be processed at the scheduled time.
If you want to use the vim, emacs or nano editor for editing cron
jobs, then, set the following in your environment and in your
~/.bash_profile
file:
EDITOR=vim
Then, to edit the default cron job, as root enter this:
# crontab -e
If this is the first time you are editing the crontab, you may want to put in the following header as a reminder or a template of the format:
#MINUTE(0-59) HOUR(0-23) DAYOFMONTH(1-31) MONTHOFYEAR(1-12) DAYOFWEEK(0-6)
#Note that 0=Sun and 7=Sun
#14,15 10 * * 0 /usr/bin/somecommmand >/dev/null 2>&1
The sample “commented out command” will run at 10:14 and 10:15 every
Sunday. There will be no mail sent to the user because of the
>/dev/null 2>&1
entry.
To list all cron jobs enter:
# crontab -l
If you’re logged in as root you can view or edit the crontab entries of any of the users:
# crontab -l -u <username>
# crontab -e -u <username>
You can also run jobs on an hourly, daily, weekly or monthly basis by adding shell scripts into one or more of the following directories:
- /etc/cron.hourly/
- /etc/cron.daily/
- /etc/cron.weekly/
- /etc/cron.monthly/
You can find out more information on the crontab manpage.
Ansible Cheatsheet
Ping all hosts:
$ ansible all -i hosts -m ping
Ping a specific host:
$ ansible raspberry-pi -m ping
Check uptime on all hosts:
$ ansible -m shell -a 'uptime' all
Check uname on all hosts:
$ ansible -m shell -a 'uname -a' all
Run playbook on more than one host:
$ ansible-playbook playbook1.yml -l 'host1.com,host2com'
Enable Debug and Increase Verbosity in Ansible
$ ANSIBLE_DEBUG=true ANSIBLE_VERBOSITY=4 ansible-playbook playbook.yml
Mounting an ISO Image as a File System
If you need to get one or more files from an ISO image and don’t have a CD/DVD drive you can mount the ISO image to a directory and navigate to the file(s) you need using a terminal or file explorer. You’ll have to do this as root.
# mkdir /mnt/iso
# mount -o loop -t iso9660 /home/user/Downloads/Custom-Build/custom.iso /mnt/iso
mount: /mnt/iso: WARNING: source write-protected, mounted read-only.
# cd /mnt/iso/
# ls
DateBuilt System/ boot/ boot.catalog efi.img initrd linux mach_kernel slint.txt
#
When you’re done just unmount the location:
# umount /mnt/iso/
#
Using sSMTP on Slackware 15.0
How to Install and Use ssmtp to Send Emails from the Linux Terminal
ssmtp is a simple utility that can be used to send emails from a Linux system to a specified email address. It is just an MTA (Mail Transfer Agent); it does not receive mail, expand aliases, or manage a queue. Instead, it forwards automated emails to an external email address. This can be useful for forwarding system alerts or other automated emails from your system to an external email address.
According to the SlackBuild comments:
sSMTP, replaces sendmail on workstations that should send their mail via the departmental mailhub from which they pick up their mail (via pop, imap, rsmtp, pop_fetch, NFS… or the like). This program accepts mail and sends it to the mailhub, optionally replacing the domain in the From: line with a different one.
Normally, when using ssmtp, you want to remove the OS’s MTA package(s), sendmail and/or postfix. In this case, ssmtp will be symlinked to /usr/sbin/sendmail, and software that sends mail shouldn’t have to be modified. If you keep sendmail/postfix installed, there’s no conflict, but any software that sends mail will have to be configured or modified to use /usr/sbin/ssmtp.
NOTE: After installing, you’ll want to edit “/etc/ssmtp/ssmtp.conf”. There’s a man page for it (man ssmtp.conf).
In this post, we’ll show you how to install and use ssmtp to send an email from the command-line interface.
Step 1: Install ssmtp
By default, the ssmtp package is not included in the default install of Slackware but you can install it by running the following command if you use sbotools:
# sboinstall ssmtp
Step 2: Configure ssmtp
Next, you’ll need to define your Gmail or other SMTP servers in the
ssmtp configuration file. You can do this by editing the
/etc/ssmtp/ssmtp.conf
file:
vim /etc/ssmtp/ssmtp.conf
Add the following lines:
UseSTARTTLS=YES
root=user@your-isp.com
mailhub=smtp-mail.your-isp.com:587
rewriteDomain=192.168.1.12
hostname=192.168.1.12
FromLineOverride=YES
AuthUser=user@your-isp.com
AuthPass=your-email-password
AuthMethod=LOGIN
Save and close the file when you’re finished. ssmtp is now configured to use your SMTP server address to send an email.
Also modify the revaliases file in the same directory. A reverse alias gives the From: address placed on a user’s outgoing messages and (optionally) the mailhub these messages will be sent through. Example:
root:jdoe@isp.com:mail.isp.com
Messages root sends will be identified as from jdoe@isp.com and sent through mail.isp.com.
$ cat /etc/ssmtp/revaliases
# sSMTP aliases
#
# Format: local_account:outgoing_address:mailhub
#
# Example: root:your_login@your.domain:mailhub.your.domain[:port]
# where [:port] is an optional port number that defaults to 25.
user:user@your-isp.com:smtp.your-isp.com:587
Step 3: Send an Email with ssmtp
First, create a text file and write some content:
vim file.txt
Add the following lines:
Subject: This is Subject Line
Email content line 1
Email content line 2
Save and close the file, then send an email with attachment
file.txt
to the external address
user@your-isp.com
:
ssmtp -v user@your-isp.com < file.txt
If everything is fine, you should see output indicating that the email was sent successfully.
And that’s it! You’ve successfully installed and used ssmtp to send an email from the Linux terminal. This can be a useful tool for sending automated emails from your system to an external email address.
How To Get Information on Linux
The manpages are historically the place to go for any information about the software and utilities installed on a Linux machine. I believe that the info page seems to be the best place for newbies to go because it is so much easier to navigate, and, like manpages, it is a still a great place to get information about your system.
Suppose you want to get information about socket. You would type:
$ info socket
and you’d get a page that looks similar to a manpage but it can be navigated by using keyboard shortcuts:
q
- exits back to the shell
u
- moves up one level
n
- moves to the next node on the current level
p
- moves to the previous node on the current level
space
- goes to the selected node under the cursor
H
- lists all of the Info commands
The commands whatis and whereis are also very useful in finding information on the system.
$ whatis printf
printf (3) - formatted output conversion
printf (1) - format and print data
The whatis command will list the set of manpages for a
specific keyword. The keyword printf is used in the
bash shell as well as the c programming language. Here
you see separate pages for each usage. To view them you would specify
the section number in your command to man, i.e.,
$ man 3 printf
or man printf
(which defaults
to section 1). This is the same output as entering
$ man -f printf
.
The whereis command will list which system files reference a specified keyword or command. For example,
$ whereis socket
socket: /usr/man/man7/socket.7.gz /usr/man/mann/socket.n.gz /usr/man/man2/socket.2.gz
$ whereis printf
printf: /usr/bin/printf /bin/printf /usr/man/man3/printf.3.gz /usr/man/man1/printf.1.gz
In case you aren’t really sure of what you’re looking for you can use a ‘keyword’ search to the man or apropos commands which will use the keyword you enter as a regex (regular expression) to find all of the matches throughout the manpages:
$ man -k printf
aa_printf (3) - print text to AA-lib output buffers.
asprintf (3) - print to allocated string
ber_printf (3) - OpenLDAP LBER simplified Basic Encoding Rules library routines f...
BIO_printf (3) - formatted output to a BIO
BIO_snprintf (3) - formatted output to a BIO
BIO_vprintf (3) - formatted output to a BIO
BIO_vsnprintf (3) - formatted output to a BIO
buffer_printf.c (3) - Buffer management functions.
curl_mprintf (3) - formatted output conversion
dprintf (3) - formatted output conversion
format (n) - Format a string in the style of sprintf
fprintf (3) - formatted output conversion
etc...
In case you are interested, the manpages are broken down into different sections as listed here:
1 General Commands
2 System Calls and Error Numbers
3 C Libraries
3p perl
4 Devices and device drivers
5 File Formats and config files
6 Game instructions
7 Miscellaneous information
8 System maintenance
9 Kernel internals
Tags: cli, man, apropos, whereis, whatis, motd
Locking Down SSH User Access
To secure your system, you shouldn’t allow root to login remotely.
Instead, if an admin needs to use the root account they should
login using their own account and then su -
or
sudo
to the root account as needed.
Edit the /etc/ssh/sshd_config
file and change the
following lines:
PermitRootLogin no
PermitEmptyPasswords no
Some of these properties may be commented out in the file, so all you’d need to do is remove the # sign.
Once these changes are made you should then restart ssh. On my system this is done by:
# /etc/rc.d/rc.sshd restart
After the restart your changes will be in affect. However, root can still login “from the local terminal”.
You can go a step further and restrict only specific users access to login via ssh. For example, if you wanted only user1 and user2 the ability to login with ssh you can add a line to the above file:
AllowUsers user1 user2
Everyone except these two users will be denied access via ssh.
Tags: cli, ssh, ssh-config, motd
Make a File Immutable
You can use the chattr command (as root) to make a
file “unalterable” so that even the root user cannot modify or delete it
without using the chattr command to revert the change. This is
done by using chattr’s +i
and -i
flags.
chattr changes a file’s attributes on a Linux file system.
Set the flag as root:
# chattr +i xrdp-notes.txt
# lsattr xrdp-notes.txt
----i---------e------- xrdp-notes.txt
A normal user can’t delete it:
$ rm xrdp-notes.txt
rm: cannot remove 'xrdp-notes.txt': Operation not permitted
Even root cannot delete it without changing the file attribute back:
# rm xrdp-notes.txt
rm: cannot remove 'xrdp-notes.txt': Operation not permitted
Once the flag is unset it can be deleted:
# chattr -i xrdp-notes.txt
# lsattr xrdp-notes.txt
--------------e------- xrdp-notes.txt
# exit
$ rm xrdp-notes.txt
$
Tags: cli, chattr, lsattr, motd
Set Mac Hostname
To set the host name of your Mac (in my case macbook-pro) in the command prompt, type the following command:
$ sudo scutil --set HostName "macbook-pro"
$ sudo scutil --set LocalHostName "macbook-pro"
Creating a Docker Registry
Creating a docker registry is rather straightforward, but you may need to do some tweaking of your docker installation in order to get it working as indicated below.
Let’s assume that you already have docker up and running on your machine.
We are going to set up a Docker registry on our local server. This
server will not have a secure connection and will only be used
internally, so we will need to allow this access by creating a file in
/etc/docker
called daemon.json to designate
this:
{
"insecure-registries":[
"localhost:5000"
]
}
Don’t forget to restart the docker daemon after making this change.
Now, create a docker volume that will persist your registry data when it goes offline:
# docker volume create /mnt/registry
This next part is optional. You can use whatever means you want to download and start the container: Create a script to download the image and start the container:
# mkdir /mnt/registry
# cat >start-registry.sh <<"EOF"
> docker run -d \
> -p 5000:5000 \
> --restart=always \
> --name registry \
> -v /mnt/registry:/var/lib/registry \
> registry:2
> EOF
Now start the registry (using the above script):
# ./start-registry.sh
Now, let’s test it by pushing a sample hello-world image to the registry:
$ docker tag hello-world:latest localhost:5000/hello-world:latest
$ docker push localhost:5000/hello-world:latest
The push refers to repository [localhost:5000/hello-world]
e07ee1baac5f: Pushed
latest: digest: sha256:f54a58bc1aac5ea1a25d796ae155dc228b3f0e11d046ae276b39c4bf2f13d8c4 size: 525
NOTE: If you are pushing to this registry from another server you
need to add the following to /etc/docker/daemon.json
and
restart docker before you will be able to do a push
(replace localhost with the IP address of the server that’s
running the registry):
{
"insecure-registries":[
"localhost:5000"
]
}
Now we can use curl to list the repositories in the registry:
curl localhost:5000/v2/_catalog
More information about creating a docker registry can be found here and if you’re interested in setting up authentication for your private registry you can find more information here
Tags: cli, docker, registry, motd
Using rsync to Sync Two Directories
You can sync two directories using rsync tool. The following command:
$ rsync -a -v --delete sending_directory target_directory;
will copy sending_directory to target_directory
creating a directory structure of
sending_directory/target_directory. The --delete
flag will ensure that any files that were previously in
sending_directory/target_directory that are not in
sending_directory will be deleted so that the contents of the
two directories are the same.
Tags: cli, rsync, backups, motd
Move Files Older Than So Many Days with Find
You may want to clean up a directory that has files older than a certain number of days, for example, 30 days. You can do this with the find command:
To move files older than 30 days in current folder to the old folder:
$ find . -mtime +30 -exec mv {} old/ \;
Tags: cli, find, mtime, mv, motd
Build, Tag, and Push a Docker Image with One Command
You can build, tag, and push a docker image to a remote all in one command with the –quiet parameter on docker build, which only outputs the SHA256 hash (annoyingly prefixed with SHA256:, but nothing a little cutting can’t fix).
$ docker tag $(docker build . -q | cut -c 8-999) myregid/imagename:1.0.1 && docker push myregid/imagename
Vim Tips: Searching a File for Multiple Strings
In Vim you can perform a search like this: /string1/;/string2
This will will combine two searches. It tells Vim:
- Search for string1, then
- From there search for string2 on any line that also contains string1.
Using RClone to Mount Your OneDrive Folder in Linux
If you have a Microsoft Office 365 account you know that it comes with 1TB of storage. You can use RClone to mount your OneDrive locally on your Linux machine so that you can read and write documents stored there.
- Install rclone from whatever package repository that your Linux operating system uses.
- Follow the instructions here to configure your OneDrive environment. The instructions are rather straightforward and a couple pieces of extra advice follows:
In the first section, type n
for a new
installation and give your mountpoint a name:
$ rclone config
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q>
n
name> remote
In the next section just press enter to accept the default:
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Now you have to choose the vendor of the service you’re configuring. There are a lot of selections hereso just look for Microsoft OneDrive, in my case it was 26:
Choose a number from below, or type in your own value
1 / 1Fichier
\ "fichier"
2 / Alias for an existing remote
\ "alias"
etc.
25 / Microsoft Azure Blob Storage
\ "azureblob"
26 / Microsoft OneDrive
\ "onedrive"
27 / OpenDrive
\ "opendrive"
28 / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
\ "swift"
Storage> 26
The next couple of selections are just the default, so press the Enter key:
OAuth Client Id
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_id>
OAuth Client Secret
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret>
Now select the global choice:
Choose national cloud region for OneDrive.
Enter a string value. Press Enter for the default ("global").
Choose a number from below, or type in your own value
1 / Microsoft Cloud Global
\ "global"
2 / Microsoft Cloud for US Government
\ "us"
3 / Microsoft Cloud Germany
\ "de"
4 / Azure and Office 365 operated by 21Vianet in China
\ "cn"
region> 1
Select the default here:
Edit advanced config? (y/n)
y) Yes
n) No (default)
y/n> n
Select the default here:
Remote config
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine
y) Yes (default)
n) No
y/n> y
Now the browser should open and you should provide your Microsoft credentials:
If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=<some long key here>
Log in and authorize rclone for access
Waiting for code...
Got code
Select 1 for Personal or Business:
Choose a number from below, or type in an existing value
1 / OneDrive Personal or Business
\ "onedrive"
2 / Root Sharepoint site
\ "sharepoint"
3 / Sharepoint site name or URL (e.g. mysite or https://contoso.sharepoint.com/sites/mysite)
\ "url"
4 / Search for a Sharepoint site
\ "search"
5 / Type in driveID (advanced)
\ "driveid"
6 / Type in SiteID (advanced)
\ "siteid"
7 / Sharepoint server-relative path (advanced, e.g. /teams/hr)
\ "path"
Your choice> 1
Select the drive number from the list, there will probably be only one to choose from, and then select the default:
Found 1 drives, please select the one you want to use:
0: (personal) id=<id key>
Chose drive to use:> 0
Found drive 'root' of type 'personal', URL: https://onedrive.live.com/?cid=<id key>
Is that okay?
y) Yes (default)
n) No
y/n> y
Now rclone will list out the configuration information and ask for confirmation. If all looks good then select y and you’re done:
--------------------
[remote]
type = onedrive
region = global
token = <some long token string here>
drive_id = <some long token string here>
drive_type = personal
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:
Name Type
==== ====
remote onedrive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
- Now you should be able to list the contents of the OneDrive directory on the console:
$ rclone lsd remote:
-1 2023-06-23 04:15:30 3 .Trash-1000
-1 2023-07-23 15:21:03 1 Apps
-1 2017-09-20 15:34:22 0 Attachments
-1 2023-07-04 23:53:04 0 Desktop
-1 2023-07-23 14:33:10 472 Documents
-1 2023-07-22 02:43:07 135 Documents from Macbook
-1 2019-11-08 21:22:23 0 Email attachments
-1 2019-11-08 21:22:23 0 Movies
-1 2023-07-23 03:39:57 53 Music
-1 2023-07-02 16:34:04 37 Pictures
-1 2022-05-24 10:39:02 2 Public
-1 2021-08-01 13:47:30 2 SyncFusion
-1 2023-06-20 11:41:24 4 Videos
If for some reason it isn’t mounted you can mount it manually with the following command:
$ rclone --vfs-cache-mode writes mount remote: ~/OneDrive &
[2] 69267
Tags: cli, rclone, onedrive, motd
Find the IP Addresses of KVM Virtual Machines (Command Line)
To find details about the virtual network you can use these commands:
root@slacker:~# virsh net-list
Name State Autostart Persistent
--------------------------------------------
default active yes yes
root@slacker:~# virsh net-info default
Name: default
UUID: 14a90f27-9a85-42ca-b434-6ce6c142690c
Active: yes
Persistent: yes
Autostart: yes
Bridge: virbr0
root@slacker:~# virsh net-dhcp-leases default
Expiry Time MAC address Protocol IP address Hostname Client ID or DUID
------------------------------------------------------------------------------------------------------------
2023-07-22 16:18:45 52:54:00:dd:7b:62 ipv4 192.168.122.216/24 centos7-bbk -
You will find the IP address and hostname listed in the last command’s output.
Optionally, to find the network interfaces’s addresses for a running domain called centos7-bbk:
root@slacker:~# virsh list
Id Name State
-----------------------------
3 centos7-bbk running
root@slacker:~# virsh domifaddr centos7-bbk
Name MAC address Protocol Address
-------------------------------------------------------------------------------
vnet2 52:54:00:dd:7b:62 ipv4 192.168.122.216/24
root@slacker:~#
Tags: cli, kvm, virsh, networking, motd
SSH Escape Sequences
Have you ever had an SSH connection timeout on you and you’re left with what looks like a locked session. Repeatedly hitting the Enter key does nothing. It seems that there is nothing that you can do except close the console terminal session…or is there something else?
Many people are not aware that SSH has its own set of keyboard shortcuts. The solution to the above problem is to terminate the connection using the first of these shortcuts.
- Press the Enter key.
- Press the tilde followed by a period.
- Press Enter again. You should now be back at your command prompt.
Supported escape sequences:
~.
- terminate
connection (and any multiplexed sessions)
~B
- send a
BREAK to the remote system
~R
- request rekey
~V/v
- decrease/increase verbosity (LogLevel)
~^Z
- suspend ssh
~#
- list forwarded
connections
~&
- background ssh (when waiting for
connections to terminate)
~?
- this message
~~
- send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)
It is important to note that you always press the Enter key before typing the key sequences above.
Tags: cli, ssh, shortcuts, motd
View lshw Output as a HTML Page
You can use the lshw command to output your hardware configuration to an html file by passing the -html parameter to it:
$ sudo lshw -html >hardware.html
List TCP Connections Sorted By Host and Most Connections
Assuming your system still has netstat installed (Slackware 15.0 does :^), you can summarize the TCP connections on you host using the following command:
$ netstat -ntu|awk '{print $5}'|cut -d: -f1 -s|sort|uniq -c|sort -nk1 -r
3 52.50.230.xxx
3 104.18.27.xxx
3 104.18.26.xxx
2 205.166.94.xx
2 192.168.1.xx
2 142.251.40.xxx
2 104.18.13.xx
1 74.120.9.xxx
1 66.255.245.xxx
1 54.154.65.xxx
1 52.96.182.xxx
1 45.56.116.xxx
1 45.33.73.xxx
1 34.117.65.xx
1 20.190.135.xx
1 192.168.122.xxx
1 192.168.1.xx
1 172.253.63.xxx
1 162.159.61.x
1 162.125.21.x
1 142.251.40.xxx
1 142.251.32.xxx
1 142.251.16.xxx
1 127.0.0.x
Tags: cli, networking, netstat, motd
Test If a Port is Open with Bash
If netcat isn’t available on your machine and you don’t have the priviledge to install it you can use this trick to test if a port is open or not. It will throw a connection refused message if a port is closed.
$ : </dev/tcp/127.0.0.1/80
And you can use it in a script like this:
(: </dev/tcp/127.0.0.1/80) &>/dev/null && echo "OPEN" || echo "CLOSED"
Tags: cli, networking, bash, motd
Recover Your Wi-Fi Password from Windows CLI
In case you misplaced your wi-fi password you can recover it very easily using 2 commands on Windoze:
Open the Terminal or PowerShell
PS C:\Users\user> netsh wlan show profile
The output will be similar to this. You need to obtain the User Profile of the connection that you’re interested in:
Profiles on interface Wi-Fi:
Group policy profiles (read only)
---------------------------------
<None>
User profiles
-------------
All User Profile : ROUTER21
All User Profile : 4YWD8-5G
All User Profile : 4YWD8
OK, so we know we have 3 profiles. Run the next command with the profile you’re interested in:
PS C:\Users\user> netsh wlan show profile name="ROUTER21" key=clear
You’ll get a bunch of output, but what you’re interested in is the field named Key Content in the Security settings section which holds the wi-fi password in cleartext:
Profile ROUTER21 on interface Wi-Fi:
:
blah blah blah
:
Connectivity settings
---------------------
Number of SSIDs : 1
SSID name : "ROUTER21"
Network type : Infrastructure
Radio type : [ Any Radio Type ]
Vendor extension : Not present
Security settings
-----------------
Authentication : WPA2-Personal
Cipher : CCMP
Authentication : WPA2-Personal
Cipher : GCMP
Security key : Present
Key Content : mywifitpassword
:
blah blah blah
:
:
Tags: cli, wi-fi, networking, motd
Prefix The Output of Any Command with a Timestamp
You can prefix the output of every line with a timestamp by piping the command to the following sed command:
$ command | sed "s/^/\[
date +“%Y%m%d%H%M%S”]/"
Create a QEMU/KVM Virtual Machine from the Command Line
You can use a combination of command line tools to create and configure a virtual machine. Here we will use few tools from the QEMU and libvirt packages to do this.
Use QEMU to create a 15GB QCOW disk image:
$ qemu-img create -f qcow2 /home/user/KVM/CentOS-Stream-9.qcow2 15G
Formatting '/home/user/KVM/CentOS-Stream-9.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=16106127360 lazy_refcounts=off refcount_bits=16
Start the installation:
$ sudo virt-install --name=CentOS-Stream-9 --vcpus=1 --memory=1024 --location=/home/user/Downloads/CentOS-Stream-9-20230704.1-x86_64-boot.iso --os-variant=centos8 --network bridge:virbr0 --disk path=/home/user/KVM/CentOS-Stream-9.qcow2 --disk size=15
Password:
WARNING Requested memory 1024 MiB is less than the recommended 1536 MiB for OS centos8
Starting install...
Retrieving 'vmlinuz' | 0 B 00:00:00 ...
Retrieving 'initrd.img' | 0 B 00:00:00 ...
Allocating 'CentOS-Stream-9.qcow2' | 2.5 MB 00:00:03 ...
WARNING Overriding memory to 3072 MiB needed for centos8 network install.
Creating domain... | 0 B 00:00:00
Running graphical console command: virt-viewer --connect qemu:///system --wait CentOS-Stream-9
Once this gets to this point another window should open up by the
virt-viewer
application (which you should install if you
didn’t already) and you can complete the installation and reboot. You’ll
use the same virt-viewer
window to use the shell or desktop
(depending upon the distro you installed).
When you are done and close the virt-viewer
GUI you can
find the VM in the running state using virsh
:
$ sudo virsh list --all
Password:
Id Name State
------------------------------------
5 CentOS-Stream-9 running
- slackware-current shut off
Then you can shut it down:
$ sudo virsh shutdown 5
Password:
Domain '5' is being shutdown
$ sudo virsh list --all
Password:
Id Name State
------------------------------------
- CentOS-Stream-9 shut off
- slackware-current shut off
Optionally, you can delete the VM domain and the QCOW file using the
undefine
parameter:
$ sudo virsh undefine --domain CentOS-Stream-9 --remove-all-storage
Password:
Domain 'CentOS-Stream-9' has been undefined
Volume 'vda'(/home/user/KVM/CentOS-Stream-9.qcow2) removed.
Using passwordgen On Slackware
The well known ss64.com site has a lot of useful tips and tricks for **nix* users and they also have a script that you can download which will create and maintain passwords for you.
If you go to their command line section
you can use the link to their GitHub site and download it for yourself.
However, when I followed their installation instructions I ran into
difficulty because the script has a dependency on pbcopy
or
gclip
(which wasn’t documented) and since I run Slackware
15.0 the copy to clipboard feature failed.
The workaround for this is to install either xclip
or
xsel
from Slackbuilds.org and make a minor
modification to the script as described here:
22c22,23
< which gclip pbcopy | grep -v -m 1 'not found'
---
> which xclip xsel | grep -v -m 1 'not found'
>
52c53
< ss64pwd_to_clipboard $one_arg
---
> ss64pwd_to_clipboard -selecton clipboard $one_arg
Tags: cli, password-managers, motd
Rename Files That Start With a Special Character
Suppose you find that you have a file with a special character and you want to delete it:
$ ls
-badfile.txt PrintHood reg57.txt
Favorites Recent scripts
$ rm -badfile.txt
rm: invalid option -- 'b'
Try 'rm ./-badfile.txt' to remove the file '-badfile.txt'.
Try 'rm --help' for more information.
$ ls *.txt
ls: invalid option -- 'e'
Try 'ls --help' for more information.
First, find the inode
of the file by using
ls -i
on the command line:
$ ls -i
54804119 -badfile.txt 56634824 PrintHood
56634825 Recent 56634807 Favorites
54804251 reg57.txt 56634833 scripts
The “-i” flag will display the file’s inode:
54804119
-badfile.txt
The inode for the “bad” file is 54804119. Once the inode is identified, use the find command to rename the file:
$ find . -inum 54804119 -exec mv {} NewName \;
$ ls NewName
NewName
Now you can delete it.
$ rm NewName
Tags: cli, rename, find, inode, motd
scp’ing Files using a Text File List
If you have a collection of files that you want to transfer to a
remote server and they are a subset of files in the current directory or
scattered among different directories you can create a list of files and
then pipe the list to scp
:
$ cat filelist.txt | xargs -i scp {} user@remote:~/backup/
How to recursively find the latest modified file in a directory
find . -type f -printf '%T@ %p\n' | sort -n | tail -1 | cut -f2- -d" "
Tags: cli, find, sort, tail, cut, motd
Bind to a Remote Port Using SSH
If you are trying to access the web page of an application running on a remote machine and you find that you are blocked, you can bind to it using SSH with similar parameters to this:
$ ssh pi@raspberrypi.local -L 8384:127.0.0.1:8384 -N
Where:
pi@raspberrypi.local
is the remote server,8384
is the port number on the remote that you wish to connect with,127.0.0.1:8384
is the local machine and the port that you want to redirect to, and-N
is a flag tellingssh
not to execute a remote command.
Using Tar with a Text Input File
If you have a lot of files in a directory and you only need to
tar
a subset of them you can create a list of the files you
want in a text file and pass it to the tar
command like
this:
$ tar -cvf tarball.tar -T filelist.txt
or
$ tar cvf tarball.tar $(cat filelist.txt)
Command Line Redirection
Redirection is very significant in shell scripting. It provides you a
means to save the output of a command to a file or multiple files (one
for stdout
and one for stderr
).
Below is a table of simple redirections that are the most useful in shell scripting. Here we are using the following naming conventions:
stdout
– The output of the script/commandstderr
– The errors generated by the script/commandoutfile
– A target filename where you wish to store the outputerrfile
– A target filename where you wish to store the errors
Command Description/Purpose command 2>errfile Redirect stderr to errfile command >outfile 2>errfile Redirect stderr to file named errfile and stdout to file named outfile command &> outfile Redirect stderr and stdout to outfile command 2>&- Just suppress error messages. No file created. No error message displayed on screen command 2>&1 Redirect error messages to standard output. Useful in shell script when you need to forcefully display error messages on screen
Tags: cli, bash, scripting, redirection, motd
Backup Files with Specific File Extension
Here is how you can use rsync
to backup only the
.ogg files in the Music directory to the OGG
directory:
$ rsync -a --prune-empty-dirs --include='*/' --include='*.ogg' --exclude='*' Music/ OGG/
Tags: cli, rsync, backups, motd
Some sed Tips
Put an * between the number 5 and 6
$ echo '12345678901234567890'| sed -e 's/^.\{5\}/&*/'
12345*678901234567890
Put a comma between each number
$ echo "1234567890" |sed -e 's/./&,/g' -e 's/,$//'
1,2,3,4,5,6,7,8,9,0
Put comma after every 2, if not even then last number exist by itself
$ echo "1234567890" |sed -e 's/../&,/g' -e 's/,$//'
12,34,56,78,90
Double space the data in file data1
$ sed G data1
Triple space the data file file data1
$ sed 'G;G' data1
Single space a double spaced file
$ sed 'n;d' data1
DOS to Linux conversion
$ sed 's/.$//' DOSfile >Linuxfile
Truncate everything after a substring (‘with’ in this case):
$ cat file.txt
Brown Potatoes with Cheese
Yellow Potatoes with Sugar and Cheese
$ sed -i.bak 's/with.*//g' file.txt
$ cat file.txt
Brown Potatoes
Yellow Potatoes
Red Hat Subscription Service
Once you have installed RHEL 9, register your RHEL subscription by running the following command on the terminal. The username and password are the login details to your Red Hat account.
$ sudo subscription-manager register --username=username --password=password
To confirm that the system is registered to Red Hat Subscription Management (RHSM), run the command:
$ sudo subscription-manager list --installed
Then refresh and subscribe:
$ sudo subscription-manager refresh
$ sudo subscription-manager attach --auto
Or, list the subscriptions that are available:
$ subscription-manager list --available --all
$ subscription-manager attach --pool=<POOL_ID>
Unregistering a system:
$ subscription-manager remove --all
$ subscription-manager unregister
$ subscription-manager clean
Speed Up DNF Package Manager
You can increase the download speed for installing packages using DNF by updating the maximum number of simultaneous package downloads.
Start by editing the /etc/dnf/dnf.conf
file:
$ sudo nano /etc/dnf/dnf.conf
Add the following line to enable DNF parallel downloads:
max_parallel_downloads=10
Another option is to select the fastest mirror from the Fedora public mirrors by adding the following line:
fastestmirror=True
Tags: cli, dnf, fedora, rhel, package-managers, motd
Creating a Python requirements.txt File
This can be done quickly by typing:
$ pip freeze >requirements.txt
at the command line.
To install packages using a requirements
file just use
pip
:
$ pip install -r requirements.txt
To check if the packages in your requirements.txt
file
are up-to-date use:
$ pip list --outdated
and to update those packages:
$ pip install -U <packagename>
This process has some drawbacks as packages are sometimes included that are no longer needed and should only be used initially and then maintained manually.
Tags: cli, python, pip, requirements, motd
Python: Initializing a New Project using venv
Start out by creating a new directory or cloning a repository (which creates a directory) and cd into it:
$ git clone https://gitlab.com/username/proj-name
$ cd proj-name/
Initialize the virtual environment:
$ python3 -m venv venv
$ source venv/bin/activate
Install whatever dependencies you need:
$ pip install package1 package2
Then develop and/or run your program:
$ python data_model.py
$ python app.py
$ sqlite3 hpr.sqlite "select count(*) from users"
When you’re done working you can deactivate the environment by typing:
$ deactivate
Tags: cli, python, venv, virtual, environment, motd
Printing Numbers using Thousand Separators
You can use a pipe to awk
to output numbers with
thousands separators (commas). For Example, here’s how you can total the
5th column of the ls -l
command and print it with thousands
separators:
$ ls -l | awk '{total = total + $5}END{print total}' | LC_ALL=en_US.UTF-8 awk '{printf("%'"'"'d\n", $0) }'
21,387
This can be adapted to other commands as necessary.
Ubuntu: Disable cloud-init on Startup
In systems that use systemd
and have a current (17.0+)
version of cloud-init
, upstream documentation describes the
process for disabling cloud-init
with either of the
following:
* touch /etc/cloud/cloud-init.disabled
or,
* add cloud-init=disabled to the kernel command line.
Tags: cli, ubuntu, cloud-init, motd
Red Hat Subscripion Manager - Registering
Registering your RHEL instance with Red Hat is straightforward assuming that you already have an account:
# subscription-manager register --username <your-rh-user> --password <password>
List the available repos or just go with a default:
# subscription-manager list --available
# subscription-manager attach --auto
Then you can do your update:
# yum update
RHEL QCOW Image Password
Did you know that although you can install RHEL from an
ISO
, you can also download a QCOW2
image from
Red Hat. However, when you try to start it up under
virt-manager
you’ll see that it boots to a
root
prompt and you don’t know what the password is. This
can be fixed by running the following from the host’s terminal:
# virt-customize -a <qcow2-image-file-name> --root-password password:<password>
Find System Hogs and Child Processes
There are a lot of tools out there that can help you to do this that are both GUI or command line-based but sometimes you’re on a production server that may have limited tools and sometimes the simplest tools are best in cases like this.
List the processes that are not owned by you that are using a lot of CPU:
$ ps aux --sort=-%cpu | grep -m 8 -v
whoami
or by memory:
$ ps aux --sort=-%mem | grep -m 8 -v
whoami
or by longest running process:
$ ps aux --sort=-time | grep -m 8 -v
whoami
List processes by a specific user:
$ ps -U gszumo
or multiple users:
$ ps -U gszumo -U katy
You can see child proceses:
$ ps -eo pid,args --forest
or children of a specific process:
$ ps -f --ppid 4775
I don’t remember where I found these tips but they have come in handy on many occasions and I am documenting them here hoping that readers (whatever few there are) will also benefit from them.
Tags: cli, ps, processes, motd
How to List Installed Packages in Fedora, RHEL, CentOS
Listing installed packages with YUM:
$ sudo yum list installed
To display details about a package:
$ yum info nginx
You can cat all installed packages to a file, copy the file to another machine and duplicate the system:
$ sudo yum list installed > installed-packages.txt
$ sudo yum -y install $(cat installed-packages.txt)
List installed packages with RPM:
$ sudo rpm -qa
List them by installation date:
$ sudo rpm -qa --last
Display information about a package:
$ rpm -qi nginx
Tags: cli, rpm, yum, rhel, fedora, motd
Git: Deleting Multiple Commits
Let’s say that you want to delete the last 3 commits that you’ve
already pushed to the remote repository. In this example, you want
566dab6
to be the new HEAD revision.
$ git log --pretty=oneline --abbrev-commit
57bc36b (HEAD -> master, origin/master, origin/HEAD) 3nd set of bad entries on 6th commit
dfb4bd3 2nd set of bad entries on 5th commit
0fd1e16 First set of bad entries on 4th commit
566dab6 Yet more good entries on third commit
d50370a More good entries on second commit
b5fbc6d Good entries on first commit
2cad4c7 Initial commit
You can use git reset
, git revert
, or
git checkout
to achieve this goal.
Using reset:
$ git reset --hard HEAD~3
$ git log --pretty=oneline --abbrev-commit
566dab6 (HEAD -> master) Yet more good entries on third commit
d50370a More good entries on second commit
b5fbc6d Good entries on first commit
2cad4c7 Initial commit
Then you have to force push to the origin:
$ git push --force
Using revert:
$ git revert --no-commit 57bc36b
$ git revert --no-commit dfb4bd3
$ git revert --no-commit 0fd1e16
or do all 3 at once:
$ git revert --no-commit HEAD~3..
Then do the commit:
$ git commit -m "Fixed commits"
$ git log --pretty=oneline --abbrev-commit
0da0f42 (HEAD -> master) Fixed commits
57bc36b 3nd set of bad entries on 6th commit
dfb4bd3 2nd set of bad entries on 5th commit
0fd1e16 First set of bad entries on 4th commit
566dab6 Yet more good entries on third commit
d50370a More good entries on second commit
b5fbc6d Good entries on first commit
2cad4c7 Initial commit
Using checkout:
Checkout that revision over the top of local files
$ git checkout -f 566dab6 -- .
$ git commit -a
$ git log --pretty=oneline --abbrev-commit
0da0f42 (HEAD -> master) Fixed commits
57bc36b 3nd set of bad entries on 6th commit
dfb4bd3 2nd set of bad entries on 5th commit
0fd1e16 First set of bad entries on 4th commit
566dab6 Yet more good entries on third commit
d50370a More good entries on second commit
b5fbc6d Good entries on first commit
2cad4c7 Initial commit
Tags: cli, git, source-control, motd
Deleting Blank Lines Using Grep
If you have a file (or a stream) that has blank lines you want to remove, you can use this command:
$ grep -v "^ *$" file-with-blanks.txt > output-file.txt
where file-with-blanks.txt
is the input file and has
blank lines and output-file.txt
will be the output file
without the blank lines.
You have to keep the space
between the ^
and the *$
otherwise blank lines which include spaces will
not be removed.
Convert webm to mp4
You can use ffmpeg to convert webm to mp4 on the command line very easily:
$ ffmpeg -i file.webm -c copy file.mp4
This will stream copy (re-mux) and avoid re-encoding.
Tags: cli, ffmpeg, webm, mp4, convert, video, motd
Parsing Bitwarden CLI
If you use the excellent Bitwarden app for you password storage you
may not be aware that they also have a command line version on Linux. It
outputs data using the JSON format. You can install the jq
JSON parser and pipe the output of the CLI’s query to it in order to
extract the username and password for a specific site in your
search:
$ bw list items --search accounts.google.com | jq '.[] | .login.uris[].uri,.login.username,.login.password'
"https://accounts.google.com"
"john.doe@gmail.com"
"** password1 **"
"https://accounts.google.com"
"jdoe@gmail.com"
"** password2 **"
Tags: cli, bitwarden, json, password-managers, motd
Export VirtualBox VDI using CLI
Sometimes you may want to move a virtual machine in VirtualBox from one server to another. Once way of doing that is to export it from the command line.
- Locate the virtual machine that you want to export (I’ll use the name UbuntuServer for the one to be exported and name the new one UbuntuServerNew), and then
- Run the export command as follows:
$ vboxmanage export UbuntuServer -o UbuntuServerNew.ova
Tags: cli, virtualization, virtualbox, motd
Bulk Change of File Extensions in Directory
If you have a directory of files where you want to change all of the file extensions you may find this code snippet useful to rename them:
for i in *.tmp
do
mv ${i} ${i%.*}.txt
done
Rename Files Downloaded Using youtube-dl
If you have used youtube-dl
to download content from
YouTube then you know that the file names can get to be rather long. In
addition to the video’s title there is a dash followed by what appears
to be a random sequence of characters just before the file extension. If
you find this to be annoying then here is a code snippet that you can
use to remove it.
This will remove the last field separated by a ‘-’ and replace it with the file extension of your choice, .mp4 is used in this example:
#!/bin/bash
for i in *.mp4
do
# Remove everything from the last hyphen to the end
file="${i%-*}"
# Rename the file and remove the space between 'file'
# and the file extension on the fly
mv "${i}" "${file%%*( )}.mp4"
done
Tags: cli, youtube, renaming, motd
Sending Email from the Command Line with mutt
If you prefer using mutt
as your command line email
tool, then here are a couple of examples of using mutt
to
send emails which can be automated quite easily to use in scripts.
Sending a file attachment (-a) and an empty message body:
$ mutt -s "System Logs" -a /var/log/somelog.log -- username@example.com < /dev/null
Sending a message body using redirection:
$ mutt -s "Email Subject" username@example.com < email.html
Sending a file attachment along with a message body:
$ echo $message | mutt -s "Email Subject" -a attachment.txt -- username@example.com
Sending Email from the Command Line with cURL
Let’s assume that you had a RFC 5822 formatted file named
mail.txt
as follows:
From: Some Sender <some.sender@gmail.com>
To: Some Recipient <some.recipient@example.com>
Subject: Saying Hello
Date: Tue, 6 Jun 2023 04:15:06 -0100
Message-ID: <1234@local.machine.example>
This is a message just to say hello.
So, "Hello".
You can forward this to a email recipient using Google’s SMTP with
the following curl
command provided you have a Google app
password:
$ curl --url 'smtps://smtp.gmail.com:465' --ssl-reqd \
--mail-from 'some.sender@gmail.com' \
--mail-rcpt 'some.recipient@example.com' \
--upload-file mail.txt \
--user 'some.sender@gmail.com:your-gmail-app-password'
Output should be something like this:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 12 0 0 100 12 0 10 0:00:01 0:00:01 --:--:-- 10
$
Transfering Files Using netcat
You can use netcat
to transfer a file from one machine
to another machine. On the receiving machine enter the following
command:
$ netcat -l 12345 > file.txt
or
$ netcat -l -p 12345 > file.pdf
Then, on the sending machine use this command:
$ netcat $MY_IP_ADDRESS 12345 < file.txt
How to Use DNF History
If you are using a Red Hat/Fedora-based distro, here are some tips on
how to use the dnf
tool to query your install history:
Retrieve a list of manually installed packages:
$ dnf history userinstalled
Retrieve a list of all transactions:
$ dnf history list all
List the changes in a particular transaction:
$ history list <num>
Undo/rollback a transaction:
$ dnf history undo <num>
Tags: cli, packaging, dnf, motd
Windows Terminal Keyboard Shortcuts
I have to use Windows where I work but I use the terminal as often as
possible. Microsoft’s Windows Terminal application makes that experience
tolerable. You can even think that you’re using tmux
at
times…well, not really, but you get the idea.
Here are some of the keyboard shortcuts that I use:
Split current terminal window
Alt Shift
- split pane horizontal
Alt Shift
- split pane vertical
Jump to other console
Alt <arrow key>
Resize current terminal window
Alt Shift <arrow key>
Close terminal window
Ctrl Shift W
Others that I don’t use as much
Ctrl + Shift + Number
: Open new profiles/tabs. Use this
shortcut to open a new profile or tab in Windows Terminal. Each number
represents a specific profile in the Terminal. Profiles are numbered in
the top-down form in the profile selection dropdown menu on the title
bar. For example, if the PowerShell profile is in the second position,
you should press the “Ctrl + Shift + 2” to open it.
Ctrl + Alt + Number
: Switch to a specific tab. Use this
shortcut to switch between tabs. Tabs are numbered from left to right
and start with “1.” For example, if you want to switch to the third tab,
press “Ctrl + Alt + 3.”
Ctrl + Shift + Space
: Opens profile selection dropdown
menu. You can then use the up/down arrow keys to select and open the
profile.
Ctrl + Shift + T
: Opens a new tab with the default
profile.
Ctrl + Shift + N
: Opens a new Windows Terminal
instance.
Ctrl + Shift + D
: This shortcut will duplicate or open
another instance of the current tab. However, it will not copy the
content of the original tab.
Ctrl + C
: Copy selected text. Select the text in the
Terminal and press the shortcut to copy it to the clipboard. Once
copied, you can paste it anywhere you want.
Ctrl + V
: Paste clipboard content. Pressing this
shortcut will paste the clipboard contents into the Terminal. Keep in
mind that only compatible content, like text, will be pasted. If you try
to paste incompatible content, like an image, the result will not be as
expected or intended.
Ctrl + Shift + W
: Close the current tab (not the entire
application).
Alt + F4
: Close the Windows Terminal window. If there
are multiple tabs, you might see a warning prompt. In that case, click
“Close all” to continue.
Ctrl + Shift + F
: Opens the “Find” function. It can be
used to find instances of a text or sentence in a terminal tab. This
functionality is similar to what you find in a browser or other
applications like Notepad, Word, etc.
Ctrl + Numpad Add/Minus
: Increase or decrease the text
size in the Windows Terminal tab.
Ctrl + 0
: Reset the font or text size its default
(100%).
Ctrl + Shift + Up/Down arrow
: Scroll up or down in the
Windows Terminal.
Ctrl + Shift + PageUp/PageDown
: Move to top or bottom in
the Windows Terminal.
Alt + Shift + Minus/Plus
: Split current pane
horizontally or vertically.
Ctrl + Shift + P
: Toggle command palette. It can be used
to select and execute a command or action from the available list.
Ctrl + Shift + ,
: This shortcut opens the Settings tab
in the Windows Terminal.
F11
: Toggle fullscreen.
Tags: cli, windows, terminal, shortcuts, motd
Installing Foliate Ebook Reader on Slackware 15
I don't know about you, but I love reading ebooks and I've used Foliate before on other distributions but can't find it for my OS of choice, Slackware. So, here's how to install it from source:
- Download the foliate software tarball from here
- Install appstream-glib (which provides dependency appstream-util) using sboinstall
- Install webkit2gtk (which provides dependency WebKit2) using sboinstall
- Build foliate. Instructions are on the GitHub page or just extract the tarball, cd into the directory and then execute the following on the command line:
$ tar zxvf foliate-2.6.4.tar.gz
$ cd foliate-2.6.4
$ meson build --prefix=/usr
$ ninja -c build
$ ninja -C build
$ su -c "ninja -C build install"
Tags: cli, ebooks, slackware, motd
Archive Only Files In a Directory
If you want to create a tar archive of only the files of a directory and exclude any subdirectories you can use the ls -la
command and pipe the output to awk
. However you need to remove the first 8 fields from the output and leave all of the remaining parts of the line in case there are spaces in the filename. One quick and dirty way of doing that is to set each of the 8 fields to a blank and then use sed
to trim the leading spaces. You can optionally add quotation marks around the filename in your output too.
$ ls -al | awk '$0!~/^d/ {$1=$2=$3=$4=$5=$6=$7=$8=""; printf("%s\"\n", $0)}' | sed 's/^[[:space:]]*/"/' | xargs tar cvf archive-name.tar
Tags: cli, tar, awk, xargs, sed, motd
Edit a Remote File Using Vim
To edit a file remotely using the vim editor, just do the following in a terminal window:
$ scp://username@IP-address//path/to/file.txt
The user must have access to the file being edited.
Echo File Until a Blank Line Is Reached
You can use the awk
program to search and print lines in a file. If you wanted to print a file until the first blank line is reached you can use the following command to do that:
awk '$0 ~ /^$/ {exit;} {print $0;}' somefile.txt
Bash Environment Variables
Here are some bash environment variables that are useful to know when you're using the command prompt:
$0
- name of shell or shell script
$1, $2, $3, ...
- positional parameters to script
$#
- count of positional parameters
$?
- exit status of most recent foreground task
$-
- current options that are set for the shell
$$
- PID of the current shell (not subshell)
$!
- the PID of the most recent background command
$DESKTOP_SESSION
- path to the current display manager
$EDITOR
- preferred text editor
$LANG
- current language
$PATH
- directory list to search for executables (programs)
$PWD
- current working directory
$SHELL
- current shell
$USER
- current username
$HOME
- current user's home directory
$HOSTNAME
- current name of the host
$TERM
- current terminal
Bash CLI Keyboard Shortcuts
Here are some quick keyboard shortcuts you can use at the command prompt:
ctrl-l
-- clear screen
ctrl-r
-- does a search in the previously given commands so that you don't have to repeat long command.
ctrl-u
-- clears the typing before the hotkey.
ctrl-a
-- takes you to the begining of the command you are currently typing.
ctrl-e
-- takes you to the end of the command you are currently typing in.
esc-b
-- takes you back by one word while typing a command.
ctrl-c
-- kills the current command or process.
ctrl-d
-- kills the shell.
ctrl-h
-- deletes one letter at a time from the command you are typing in.
ctrl-z
-- puts the currently running process in background, the process can be brought back to run state by using fg command.
esc-p
-- like ctrl-r lets you search through the previously given commands.
esc-.
-- gives the last command you typed.
How To Find All of the Shell Scripts In a Directory
This is a quick and dirty way which will list all of the files that are shell scripts:
for i in *
do
type=$(file ${i}|awk -F, '{print $2}')
if [[ "${type}" = " ASCII text executable" ]]; then
echo "${i} is a shell script"
fi
done
How To Use Github Tokens on the Command Line
GitHub’s access policy requires you to use tokens instead of username/password to update your repositories.
To adapt your existing local / cloned repos to token based auth:
$ git remote remove origin
$ git remote add origin https://[TOKEN]@github.com/[USER]/[REPO]
$ git push
Clone repos using token based authentication (non public repos)
$ git clone https://[username]:[token]@github.com/[accountname]/[reponame]
Tags: github, tokens, motd, cli
How To Count All The Files Extension Recursively In Linux
To count all the files by file extension recursively on the command line
$ find . -type f | sed -n 's/..*.//p' | sort | uniq -c
40 3g2
5 AVI
13 DS_Store
28 JPG
30 MOV
133 MP4
64 THM
1 docx
18 jpg
1 json
4 m3u
89 m4a
2 m4r
156 m4v
41 mkv
112 mov
38 mp3
587 mp4
1 nfo
2 osp
30 png
1 sh
4 srt
6 svg
10 torrent
6 txt
5 webm
10 zip
Tags: cli, motd, find, sed, sort, uniq
Untar a Tarball to a Remote Directory
Sometimes you may need to copy an entire directory structure to another system using the command line. Here is a quick way to do it using the tar command:
cat myfile.tgz | ssh user@host "tar xzf - -C /some/dir"
Getting Rid of ^M Line Endings in a Text File
If you have a text file that has funny looking ^M characters at the end of each line, in most cases, you have to get rid of them before they can be used. This is especially the case when you've copied or transferred a file from a Windows-based system to a *nix-based one. If these files are shell scripts meant to run on the *nix-based system they more often than not won't work. There are various solutions to this problem.
First, let's create two text files: one with ^M line endings and one without:
$ for line in 1 2 3 4 5; do echo "This is line ${line}^M" >>file1.txt; done
$ for line in 1 2 3 4 5; do echo "This is line ${line}" >>file2.txt; done
Now let's see what's different between these two text files:
$ ls -l
total 8
-rw-rw-r--. 1 gszumo gszumo 80 Oct 29 17:43 file1.txt
-rw-rw-r--. 1 gszumo gszumo 75 Oct 29 17:44 file2.txt
$ file file1.txt
file1.txt: ASCII text, with CRLF line terminators
$ file file2.txt
file2.txt: ASCII text
$ diff file1.txt file2.txt
1,5c1,5
< This is line 1
< This is line 2
< This is line 3
< This is line 4
< This is line 5
---
> This is line 1
> This is line 2
> This is line 3
> This is line 4
> This is line 5
$
What does this tell us?
- The ls command tells us the file sizes are different even though the visible text is the same.
- The file command tells us that both files are ASCII text but file1.txt has CRLF line terminators, and
- The diff command tells us that each line is indeed showing us that it's different.
So, how do we fix this?
My favorite solution is to use vi or vim interactively. There are 2 easy ways to get rid of the ^M from a single file using vim:
- Enter the command:
:%s/^M//g
on the vim command line then save the file, or - Enter the command:
:fileformat=unix
on the vim command line and save the file.
However, if you have a whole directory or directory tree full of these kinds of files using vim on each one individually will become quite tedious. For this you need the scripting capability of the command line!
The 'tr' command is one quick way of getting rid of them using the Linux or macOS command line:
cat somefile | tr -d '^M' >outputfile
We can use this as a template in order to determine whether or not the file needs to be updated:
for i in *
do
string=$(file ${i})
test "${string#*'CRLF'}" != "$string" && echo "CRLF found in ${i}"
done
If the 'echo' part of this snippet only gets called when the first part of the test is true, so then we know that the file has '^M' line endings. We have to turn the second part of the test into a script that will massage the file to remove the line endings. Here's a bash snippet that will put the two together and do the job:
for i in *
do
string=$(file ${i})
if [ "${string#*'CRLF'}" != "$string" ];then
cp ${i} ${i}.bak
cat ${i}.bak|tr -d '^M' >${i}
rm ${i}.bak
fi
done
This is a bare-bones piece of code which doesn't do any error checking, which should be added in the event that the user running this script doesn't have the necessary permissions to copy, write or remove files that match the test.
Remember, in order to create a '^M' character in the terminal hold down the CTRL key while typing vm.
You can also use 'ex' to replace a string in a file using:
ex -s -c '%s/old-str/new-str/g|x' filename.txt
Alternately, we can use sed to do the same thing. At the command line you can replace any string in a file dynamically by entering:
sed -i 's/old-str/new-str/' filename.txt
So to remove the ^M charaters just do:
sed -i 's/^M//' filename.txt
to remove the ^M characters. You can use the same for/do/done loop structure as mentioned above to iterate over multiple files:
for i in *.txt
do
sed -i 's/^M//' ${i}
done
'Nuff said.
Removing ^M Characters From Multiple Files
In my day job I find that I frequently need to remove those pesky '^M' line endings from text files that are transferred from one system to the other. Most of the time it is just one file that needs to be fixed so going into vi and typing
:%s/^M//g
solves the problem, but on occasion I might be confronted with performing this process on multiple files and sometimes recursively through many subdirectories. When that happens it becomes necessary to bring out the "big guns" and do:
$ for file in $(find /path/to/dir -type f); do
tr -d '\r' <$file >temp.$$ && mv temp.$$ $file
done
$
(BTW, to generate a ^M character using the PuTTY emulator is easy. Just press CTRL-^ followed by CTRL-M)
Of course, in Linux, UNIX and AIX shells there are many ways to skin a cat and this is just one of many possible solutions. This solution was found on a Daily Vim Blogspot site which has more very interesting tips & tutorials.
Get the Hash of a File on Different Operating Systems
Sometimes you just need to hash something. Here are some command lines to do just that.
Windows PowerShell:
Get-FileHash C:\path\to\file.iso -Algorithm [MD5|SHA1|SHA256|SHA384|SHA512|MACTripleDES|RIPEMD160]
(SHA256 is the default if the parameter is omitted)
Linux:
md5sum /path/to/file
sha1sum /path/to/file
sha256sum /path/to/file
macOS:
MD5: md5 /path/to/file
SHA1: shasum /path/to/file or shasum -a 1 /path/to/file
SHA-256: shasum -a 256 /path/to/file
Copy Entire Directory Structure Using Tar
Sometimes you need to copy an entire directory structure to another location on the command line. Here is a quick way to do it using the tar command:
tar cf - * | ( cd /target; tar xfp -)
Sort Directory by File Size
To sort files in a directory and to list them by the biggest file first:
$ du -a | sort -n -r | more
Alternatively you can use:
$ du -s * | sort -n -r | more
Alternative to Using cp
To backup files from a source directory to a backup directory:
$ mkdir /path/to/backup/directory
$ cd $HOME/source/directory
$ tar cf - . | (cd /path/to/backup/directory && tar xBvf -)
Define the following alias:
tar cvf - . | ( cd \!* ; tar xvf - )
or as an alias:
alias cpbytar='tar cvf - . | ( cd \!* ; tar xvf - )'
(The alias definition above is for Bash)
To do a recursive copy of a directory to another location, preserving the PERMISSIONS and OWNERSHIP of the files. "cd" to the source location and invoke the following alias:
cpbytar