Show Menu
Cheatography

Linux Troubleshooting Cheat Sheet (DRAFT) by

This is my approach to learn linux administration by doing some troubleshooting

This is a draft cheat sheet. It is a work in progress and is not finished yet.

System Access Troubl­esh­ooting

Server not reachable
#ping servername

if no sucess search for IP for "­ser­ver­nam­e" with
#nslookup servername

#ping IP-Ser­vername


If IP-ser­vername is pingable check:
/etc/host

/etc/r­eso­lv.conf

/etc/n­ssw­itc­h.conf

If IP-ser­vername is not pingable
ping another server with name and ip
Check if your machine has ip-adress `#ip -c -b a
get gateway-ip with
#netstat -rnv
and check destin­ation 0.0.0.0 to gateway-ip
ping gateway -ip
check local physics cable etc.


App or website not reachable
ping servername and server-ip
if success connect to service
#telnet ip port

if not success login into server check services eg.
#ps -ef |grep -i network

or
#systemctl status <se­rvi­ce>
or
'systemctl --failed


Cannot ssh as root/user
check if root is not permittet
#/etc/­ssh­/ss­hd_conf

check
#less /var/l­og/­secure

no user or nologin shell
check
#id <us­er>

check
#less /etc/p­asswd



Firewall
check
#service iptables status

or check
#systemctl status iptabl­es.s­ervice

or check
#ps -ef|grep iptable

or check
#systemctl status firewalld

try to temp disable firewall
#systemctl disable firewalld


Terminal or Client is not working
check correct port user and password syntax

Filesystem Troubl­esh­ooting

If you cannot CD into Dir or cannot open file or run a script
------­---­---­---­---­---­---­---­------
Dir/file not existing
check path relative or absolute
check permis­sions
check file type
check parent dir permissons
check for hidden dir



Trouble to find files or dir
------­---­---­---­---­---­---­---­------
command find or locate
correct syntax:
#find /path  - type [d,f] -name 'my_dir' or 'my_file'

file/dir exits?



Cannot create links
------­---­---­---­---­---­-----
which type of link hard or symbolic
syntax source first then target
check permis­sions
source or target file/dir missing or not existing



Cannot write to a file
------­---­---­---­---­---­------
check if it exists
check abs or rel path
file type
check parent dir permis­sions
check for hidden file
check for file is open by an another user
run
#getfacl




Cannot Move/R­ena­me/­Del­ete­/Copy file
------­---­---­---­---­---­---­---­---­---­---­---­---­---­------
check if it exists
check abs or rel path
file type
check parent dir permis­sions
check for hidden file
command syntax
#mv; cp;   "­Source then Target­"




Cannot change file permis­sio­ns/­own­ership
------­---­---­---­---­---­---­---­---­---­---­------
check if file exists
check abs. or rel path
check file ownership (user and group)
check Permis­sions
check parent dir permis­sions
check for hidden file
command syntax:
#chown  or chmod (user then file)

command with recursive option -R
only root can change the file ownership



Cannot view others users files
------­---­---­---­---­---­---­---­---­---­---­---­-------
check if file exists
check abs. or rel path
check file ownership (user and group)
check Permis­sions
check parent dir permis­sions
check for hidden file
command syntax
only root can view any files



Cannot change password
------­---­---­---­---­---­---­---­------
check if file exists
check abs. or rel path
check file ownership (user and group)
check Permis­sions
check parent dir permis­sions
check for hidden file
check command syntax: #passw <us­er>`
only root can change pw of any user



Disk space full
------­---­---­---­------
ckeck :
#df -h

check HD status:
#badblocks -v /dev/DISK
or
/dev/sda

check
#iostat




Delete old files
------­---­---­-------
command:
#rm

syntax:
find /path/­to/­files -type f -name '*.exa­mples' -mtime +30 -exec rm {} \;




Filesystem is corrupted
------­---­---­---­---­---­---­-------
filesystem partit­ions:
/var
;
/etc
;
/root
;
/home

check file system :
#df
,
#fdisk -l

check logs in
/var/l­og/­mes­sages
or
/var/l­og/­syslog

run
#fsck
on block device (eg /dev/sda) not on mount point
unmount filesystem the run
#fsck




Corrupted /etc/fstab
------­---­---­---­---­-------
check filesy­stem:
#df

cat or
#less /etc/fstab

system not booting: incorrect entry in fstab or fstab deleted
Steps to do:
*boot in rescue mode mounting a cd or cd-iso image
*choose option 1 to mount root fs
*fix the
etc/fstab file

*for deleted file run
#blkid


 

System Admini­str­ation Troubl­esh­ooting

Out of memory
#free -m -h

#top

#vmstat

#dmesg |grep -i "Out of memory­" 
(/var/­log­/me­ssages or /var/l­og/­syslog)
check
#/etc/­sys­ctl.conf


Add Swap Memory
$df -h

#dd if=/de­v/zero of=/ne­wswap bs=1M count=1024 

#free -m

#chmod 600 newswap

#mkswap /newswap

#swapon /newswap

change ftsab to: /newswap swap swap defaults 0 0

Delete Swap space
#swapoff /newswap

#rm /newswap

#free -m

change your fstab file

System rebooted or process is restarting
#systemctl status <pr­oce­ss>

#uptime

#top

#dmesg

#iostat -xz 1

#journ­alctl


Check system logs: /var/l­og/­mes­sages /var/l­og/­syslog /var/l­og/­boo­t.log
Check app logs

Unable to receive an IP Address
check DHCP Server
check network virtual setting
#check network hardware setting

lspci | egrep -i 'eth|w­ifi­|wi­reless'
#nmcli -p dev

#ip  -h -c address

#ifup <in­ter­fac­e>   ifconfig up <in­ter­fac­e>

#systemctl restart network

check config files /etc/s­ysc­onf­ig/­net­wor­k-s­cri­pts­/if­cfg­-enp0s3 or ifcfg-eth0

IP assigned but no reacha­bility
check to which network you are connected to use ifconfig or ip a
ping your gateway
#netstat -rnv

#ethtool or mii-tool

#ifup <in­ter­fac­e>

#systemctl resart network

#ifconfig or ip  -4 <-6> -c add

try to turn off tempor­arely the firewall

What if can't run a specific command
check permission and ownership
check abs. and rel. path
#echo $PATH

check for missing or nit installed sw
#whereis <sw>

#yum provides <pa­cke­t>   or dnf

#yum seach telnet


Password not changeable
check file /etc/p­asswd
and /etc/s­hadow( pwconv will recreate file)
user exists correct written ?
be root
#passwd <us­er>

after changes in /etc/s­hadow use pwconv (PW of each
user has to set again)

User has no home dir
dir not existing
dir exists but no entry in /etc/p­asswd
check permission ,ownership and spelling of dir

Change words in files
use vi: :1,$s/­old­wor­d/n­ewword/
use sed:
#sed -i 's/old­sri­ng/­new­str­ing/g' <fi­len­ame>


sed command
tbd

Kill Process, User, Terminal
Find process ID (PID)
#ps -ef

#kill <PI­D>

#kill -9 <pi­d>
brutal kill
#pkill <pr­ocess name>

#killall <pr­ocess name>


 root pw recovery

restart computer (physical console access nessesary)
edit grub, search for
ro
change to:
rw init=/­sys­roo­t/b­in/sh

press <ctrl x>
system reboots in single user mode
`>p­asswd root'
>change pw

`>touch /.auto­rel­able'
>exit

reboot

logged in users
#last


System running slow
Check disk space (df -h and du)
Check processing (top, free, lsmem, cat /proc/­mem­info, vmstat,
pmap <PI­D>, dmidecode, lscpu, /proc/­cpuinf)
Check disk issues (iostat -y 5, lsof)
Check networking (tcpdump -i <in­ter­fac­e> , lsof -i -P -n |grep -i listen,
netstat -plnt or ss -plnt, iftop)
Check uptime (uptime)
Check logs
check hw status log into system
Other external tools (htop, itop, iptraf, psacct)

Rollback updates and patches
Package or patch
#dnf history  

#dnf history undo <id> 

Update (upgrade does not workold obsolete packages are lost)
#dnf history undo <id>

System Recovery

 

Additional Knowledge and Tools