This document will cover the generic Unix commands that are largely the same on all flavors of Unix.

File/Dir Permissions

sticky bit, setuid, setgid

chmod 1777 /tmp      = sticky bit on dir

chmod 4755 /tmp/dir  ~= chmod u+s /tmp/dir   = setgid on dir,  content will always be owned by the primary owner of the dir.
chmod 4755 /tmp/file ~= chmod u+s /tmp/file  = setgid on file, execute with owner priv. common eg: setuid root.

chmod 2755 /tmp/dir  ~= chmod g+s /tmp/dir   = setgid on dir,  content will always be owned by the group.
chmod 2755 /tmp/file ~= chmod g+s /tmp/file  = setgid on file, execute with group priv

chmod 6755 /tmp/file ~= chmod ug+s /tmp/file

newgrp	groupname	# change the default group user will create file as 
			# only last for current session
			# Can prompt for password if group is password protected?


getfacl file1				# read permissions
setfacl -m u:andy:rw  file1		# -m = modify: give (a secondary user) named andy read/write access to file1
setfacl -m g:admin:rw file1             # g for secondary group, here a group named admin, get rw access
setfacl -x u:andy     file1		# -x = remove permissions

directory can have default acl set so that all files within it will inherit such acl automatically.  use "d" to specify it as default settings.  eg:

setfacl -m d:g:admin:rw  dir1

crontab -l = list
        -e = edit
        -r = remove  (no confirmation, everything gone for the user!!)

entries stored in /var/spool/cron
# crontab file format 
# minute hour day-of-month month day-of-week0=sun cmdline 
# 0-59   0-23 1-31         1-12  0-6              echo "Hello World!"
*/5      *    *            *     *                echo "Linux can easily define job for every 5 min" > /dev/null
/etc/crontab : good for sys admin, single file for all jobs, has extra column defining user to run job as.

# Example of job definition:
# .---------------- minute (0 - 59)
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# |  |  |  |  |
# *  *  *  *  * user-name 	command to be executed

*/5  *  *  *  * root 		/usr/local/bin/pbs_check

### can change MAILTO for rest of "script"
*/5  *  *  *  * root 		/usr/local/bin/lustre_monitor

centos 7 allow .sh files in 
/etc/cron.monthly/  which run 1 month after last run, with last run recorded in /var/spool/anacron/cron.monthly   eg 20201227

They are driven by /etc/anacrontab # eg tweak what time job run, MAILTO, ENV, etc.



-h		= brief help, no man pages needed.
-r 		= recursive
-l1  		= depth level, default to 5
--no-parent	= don't bother with ref to parent dir (-np)
-A.gif		= download only .gif files

-c		= continue previous interrupted download
-nc		= no clobber of existing files
-b 		= process job in background, returning to prompt
--passive-ftp	= use passive ftp

-k		= change link ( convert non-relative links to relative)
-m		= mirror site  options, eg, download file only if newer.
-l=20		= retrieve to depth = 20

--http-user=USER	= web page with user/login req
--convert-links		= convert the downloaded html link to local files
-Pdownloaddir		= place downloaded file in downloaddir instead of the name.

eg download only *.gif files from server
wget -r -l1 --no-parent -A.gif

eg2, ftp with username and password as username:password@site:

eg3: download ibm patches, which is dir with set of links.
wget --glob=on --retr-symlinks --level=1* 

eg4: getting suse linux from ftp site:
wget -r -m  -np -c -l10

wget obeys robot rules. in file /robot.txt

If proxy is mandatory, set the proxy server thru env var http_proxy and ftp_proxy.  eg:
setenv http_proxy ""
export ftp_proxy=""
If auth is req, wget has parameters to submit these also.  

wget --no-verbose${KNIME_VER}-${KNIME_PLAT}.tar.gz -O $KNIME_GZ 
	-O will force output to a specified filename, overwrite if exist.

curl has more power than get, especially when need to use POST instead of GET

    Test if privately hosted docker registry (v2) is responding:

    curl -k -L -XGET "https://tin:${mypass}@registry:443/v2/_catalog" 
    # return a list of images as json
    # -k skip check whether cert is valid/from known ca$

    curl -k -L -XGET "https://tin:${pwd}@registry:443/v2/conductor/tags/list" 
    # list all tags for container called "conductor"

    Other examples to try if anonymous login to registry is allowed:

    curl -k -L -XGET  "https://beagle/" -d "email=tin@place"            # do nothing?  no complain about auth either
    curl -k -L -XGET  "https://beagle/v2/_catalog" -d "email=tin@place"
    curl -k -L -XPOST "https://beagle/v2/_catalog" -d "email=tin@place"

rsync -avul source /path/to/destination
            sync ./source to /path/to/destination
            source or destination could be NFS mounted location.
            only newer files in local dir will be copied to dest
            if dest has newer file, they will not be changed.
      -a    archive mode, preserve sym link, attributes, owner/perm, etc.
      -n    dry run, use w/ -v and will list files that will be cp'ed
            w/o actually doing it.
      -l    copy sym link as sym link (like tar)
      -L    copy file refered by the sym link
      -z    use compression during transfer (when used with remote host)
	--delete	delete files in dst that is not in source
			Use this to sync/backup user /home during migration
	--preogress	show progress (useful when not in -v?)

rsync -avuL /f /cygdrive/p/backup_of_master_data_DrvF
            cmd used at home in win2003 to backup all data to removeble ide drv.
            does not support funky chinese mp3 filename :(

rsync -avl -e ssh  --rsync-path=/usr/local/bin/rsync remote-svr:/dir/* ./local/storage
	use ssh as transfer protocol with remote machine
	allow specification of where rsync is located on remote host
	either source or destination machine could be a server: kind of entry

screen -T xterm-256color -s /bin/zsh      -S z1    # set terminal type cuz Zorin setting to something weired older machine can't handle
screen -T xterm-256color -s /usr/bin/fish -S f1    # use fish shell for all new screen session
work as usual

screen -r  = resume
screen -rd = resume, detach all other sessions
screen -w  = list screens
screen -d -r  reatach remote and retach here
screen -x = allow for multiple attach to a screen (mirror!)
screen -s /usr/bin/fish = all scren sessions will use the fish shell.

Can use named screen sessions, so don't have to lookup pid using screen -ls
screen -S scr1
screen -r scr1  = resume the specified named session scr1.  Note that it does partial name match.  scr could be scr1, scr2, or just scr

ctrl-a,d       = detach
ctrl-a,ctrl-d  = detach  (ie, after ^a, could hit ^d or just d to detach)
crtl-a,x       = lock keyboard
crtl-a,ctrl-x  = lock keyboard
ctrl-a,?  = help

screen window, better think of them as tab :)
ctrl-a,c  = create new window ( or enter screen  w/in an existing screen window)
ctrl-a,n  = switch to next window
ctrl-a,p  = switch to prev window 
ctrl-a,w  = show list of window
ctrl-a,"  = prompt for window name or number to switch to
ctrl-a,0  = switch to window number 0 (to 9, enter - for blank window)
ctrl-a, A = prompt for a title name for current window
ctrl-a, N = show number and title of current window
ctrl-a, k = kill current window

ctrl-a, S = create horizontal split (region) mode # ie top/bottom split.  tmux ^b, " 
ctrl-a, | = create vertical   split (region) mode # ie left/right split.  tmux ^b, %  # newer feature? not avail on osx
ctrl-a,tab  = switch input focus to next region (split)
ctrl-a, ^I  = move focust to next split window (^I=TAB).
	splits are for the current session, each reagion can navigate to different window/tab as desired.
	the split are gone when detached.
	i have not used this much, prefering to use new terminal and create additional screen session.
	tmux approach is better, as it persist and resumable :)

ctrl-a, [ = history, allow scroll back, but really for cutting text into buffer.  
            Enter marks begin pt, move around.  Enter again marks end pt.
            /, ?, n, p = search (like in "less")

ctrl-a,a  	= send ctrl-a to the terminal
ctrl-a,b 	= send a break
ctrl-a,ctrl-b	= send a break (ie, after ^a, can hit ^b or simply b to send break.  screen's help means or, not that need to hit three seq of ^a, ^b then b to send a break)

screen & tmux side-by-side
C-x = Control-x
M-x = Meta-x, typically Alt-x
tmux use ctrl-b instead of ctrl-a that screen use, less keymap conflict
tmux use session, window, pane (screen has session, "window" (which i think of as tab)

  • Key bindings that need to memorized for effective use of tmux
    ^b ,	# rename win/tab	^b $   	# rename session/instance
    ^b w 	# list window		^b s	# list   session
    ^b c    # create window		^b: new	# new session
    ^b n	# next window		???	# jump to next session?
    ^b l	# last active window
    ^b " 	# top-bottom pane	^b %  	# side/side pane
    ^b ^o	# rotate pane		^b alt-o # rotate pane backward
    ^b q 0	# jump to pane 0
    ^b ;	# jump back to prev pane
    ^b SP   # cycle pane arrangement (alt-5 is good for quad)
    ^b z    # zoom/unzoom current pane
    ^b x  	# close/kill current pane
    ^b ^b	# send ctrl-b, useful in nested tmux or vi back page
    ^b :    # command
    ^b :set -g mouse on		# mouse mode, allow resize pane w/ tmux 2.1+
    tmux new -s tmux1			# this start a new session, like screen session # new process instance
    ctrl-b, d = detach			# -or- tmux detach
    tmux ls 				# list all sessions
    tmux attach  -t tmux1			# attach a named session (screen -r)
    tmux attach -dt tmux1			# detach remotely and attach here (screen -rd)
    ctrl-b, ?		= help
    # screen sessions/instances are independent, but tmux also provide abilities to swap between them:
    ctrl-b, s		# list sessions [ tmux process instances ]
    ctrl-b, w		= list windows  [ those within the session ]
    ctrl-b, $		# rename sessions (instance) (name shows up in tmux -ls)
    ctrl-b, ,		= rename window   (tabs)     (name appears in bottom green status bar)
    # tmux window (tab) is like screen tab
    ctrl-b, c		# crate new window (think of this as tab, same as screen)
    ctrl-b, w       	# list window (screen ctrl-a, ")   # very much like screen "tab".  new version print a tree of sessions+windows
    ctrl-b, n 		# next window (same as screen)
    ctrl-b, p 		# prev window (same as screen)
    ctrl-b, ,		# rename window
    ctrl-b, !		# convert window (tab) into pane  (!, screen can't do this)
    splitting is best thought of as pane, which is within a window (tab)
    ie, switching to a different window/tab will completely change all panes 
    (spliting pane behaviour is very different than screen's split)
    (the splits are rememberd w/in the window/tab they are created, so they get resumed, which is nice)
    ^b, "		# split/div pane vert  (screen ^a,S) # top/bottom split. ie 2 fat  and wide   panes == 2 panes with 1/2 of vertical height
    ^b, %		# split/div pane horiz (screen ^a,|) # left/right split. ie 2 tall and skinny panes == 2 panes with 1/2 of original width.  Bad for cut-n-paste
    ^b, j/k/h/l 		# move between panes, same as vi movement keys
    ^b, dw/up/lf/rg arrows	# move between panes
    ^b,^Up			# resize: make pane divider go up
    ^b,^Dn			# resize: make pane divider go down
    ^b,M-Up			# resize: make pane divider go up by 5 cells (M=meta=alt)
    ctrl-b, q		# show pane number
    ctrl-b, q, 0		# jump to pane 0
    ctrl-b, q, 1		# jump to pane 1
    ctrl-b, ;   		# jump to previous active pane
    ctrl-b, x		# kill pane (ask confirmation).  all panes are killed before the tab get killed
    ctrl-b,alt-2		# equalize all panes, arrange top to bottom.  [alt-n maybe intercepted by terminal to switch between tabs]
    ctrl-b,alt-1		# equalize all panes, arrange left to right
    ctrl-b,alt-3		# T split.  Top wide screen with many bottom small panes
    ctrl-b,alt-4		# rotated T split, with tall panel on left.
    ctrl-b,alt-5		# 4 equal quadrants. If 3 panes, get inverted T split, like mirror of "3"
    ctrl-b,SPACE		# cycle between the 5 pane arrangement options
    ctrl-b,z        	# zoom current pane (but still leave | at borders so no good for cut-n-paste
    Ctrl-b,ctrl-o		# rotate panes forward  (useful eg in alt-3 view)
    ctrl-b,alt-o		# rotate panes backward
    ctrl-b,{		# swap with prev pane (exchange position, not just cursor focus)
    ctrl-b,}		# swap with next pane (exchange position, not just cursor focus)
    			# idea is likely to swap small pane into larger pane in one of alt-3 or alt-4 arrangement
    ctrl-b,m		# mark current pane (thicker green border to make it more obvious, i like it)
    ctrl-b,M		# unmark current pane 
    ctrl-b, ~		# show prev tmux message, if any (eg error in command) (?)
    ctrl-b, ctrl-b		# send ^b (eg scroll back in less, vi; or nested tmux)
    tmux commands
    for use in command mode and .tmux.conf file
    Once inside tmux, can re-execute the config as
    tmux source-file ~/.tmux.conf
    ctrl-b, :               # enter command mode
    resize-pane -D 		# move pane divider down
    resize-pane -U 10	# move pane divider up    by 10 lines
    resize-pane -L 10	# move pane divider left  by 10 lines
    resize-pane -t 1 -R 10	# move pane divider right by 10 lines for pane id 1
    # has these that would be useful to use mouse to resize pane size, but don't work even in tmux 2.9a :(
    # not even when placed in ~/.tmux.conf
    setw -g mode-mouse off
    set -g mouse-select-pane off
    set -g mouse-resize-pane off
    set -g mouse-select-window off
    # since 2.1, just need
    set -g mouse on
    # but mouse highlight will go to tmux buffer, not X clipboard, which is usually not what I want :(
    find /home -name myfile.txt -exec ls -ld {} \;
    find options
    -user UID		: find all files owned by user UID
    -exec CMD {} \;		: {} represent the matched entry (full path)
    			  CMD will be executed with {} as arg
    			  \; is required to end exec section
    finding files: 
    find . | grep 'FILENAME' 		# not 'correct' usage, but works :)
    find . -name FILENAME -print
    find . -user root  -exec chown weblogic:weblogic {} \;
    	run in weblogic dir, replace all files owned by root to weblogic
    	(remember to change back to root the back to root)
    find . -ctime +30 -exec mv -t OLD_LOG {} \;	# move files (eg logs) more than 30 days (creation time) old to a target dir named OLD_LOG   
    						# (-t in mv puts destination dir first, then take args for files to be moved)
    find . -perm -o+w -exec ls -l {} \;
    	long listing of all files whose permission is world writable
    find . ! -user tho01 -print
    	list all files whose owner is not tho01
    other cmd: ls -l, so to see full listing of file instead of just path, 
    but any entry that is a dir will return rather long listing...  (can use  type=file only to skip dir)
    find . -type f -ctime -5 -exec ls -l {} \;		# files modified less than 5 days ago. ie NEW file w/in past 5 days.   eg script looking for new files to backup
    find . -type f -ctime +5 -exec ls -l {} \;		# files modified more than 5 days ago. ie OLD files older than 5 days. eg use in auto delete script
    find . -type f -ctime  5 -exec ls -l {} \;		# files modified exactly   5 days ago. ie EXACTLY 5 days old (seldom use).
    find . -name ZQ\*DAT -type f -ctime +5 -exec rm {} \;	# rm ZQ*DAT older than 5 days.
    find  /usr/local/jboss-6.0.0.Final/server/default/log -type f \! -name \*gz -ctime +4 -exec gzip --best {} \;
    find  /usr/local/jboss-6.0.0.Final/server/default/log -type f -ctime +30 -exec rm {} \;
    	# compress log files older than 4 days, delete everything more than 30 days old
    find /home/USERNAME -path /home/USERHOME/.snapshot -prune -o -print 
    	# find all files from the user home dir, but exclude .snapshot dir from NetApp
    	# find and .snapshot is strange, sometime it will enter .snapshot (typically full path
    	# to the "root" of the mount, other it won't (typically relative path from lower level of the tree) ??
    	# I guess it depends on how the dir was stat'd...
    find /nfshome -uid 501 -exec chown -h 8001 {} \;
    	# change UID of all files beloging to user with UID 501, change it to 8001
    	# -h option of chmod effectively leave destination of sym link with original owner
    chown -R 8001:2000 --from=501:201 .
    	# chown can do matching before it actually change ownership
    	# maybe able to archive similar result without the find command above.
    FECHA=$(date "+%Y.%m%d")



    gpg --symmetric myfile.txt		# symetric key, single password.
    gpg -e -r username myfile.txt		# encrypt using pgp public key stuff
    pgpe - Encrypts and signs messages
    -a, --armor
              Turn on "ASCII Armoring."   This  outputs  a  text-only
              version  of your encrypted text.  This makes the result
              safe for mailing, but about 30% larger.
    -c   Conventional  encrypting  mode.
            ie, symetric cipher, no public key stuff.
    pgpe -c -t test.txt
            This encrypts test.txt using conventional, single key encryption.
            produce output file test.txt.pgp
            -t = platform independent text mode.
            Note that original file will remain, must be manually removed.
    windows binary version of gnupg: (need dir c:\gnupg to exist)
    gpg -c test.txt
            encrypt file with symetry key, generate test.gpg, leave original intact.
    gpg -d test.gpg
            decrypt test.gpg, output to stdout.
    gpg -h
            display help usage
    pgpv - Decrypts and Verifies messages.
            -m : more mode.  page encrypted file using $PAGER
            This will not produce an output file.
    pgpv test.txt.pgp
            decrypt file, placing output in same filename sans .pgp extension.
    -e      : encrypt
    -c      : symetric cipher
    --decrypt [FILE]        : decrypt to stdout,
                            : read from stdin if FILE not specified
    gpg -c area51
            produces simple password encrypted file area51.gpg
            original file remains!
    gpg --decrypt area51.gpg
            decrypt file and write it to std out
    gpg --gen-key
            Generating private/public key pair (keys, db, etc are saved in $HOME/.gnupg/)
    gpg --export -a > tin-brio.gpg-public-key
            Generate a 7-bit ASCII text block of the public key, suitable for publishing on Internet
    gpg --import tin-tokyo3.gpg-public-key
            Import other user public key into the db (must have these key in db before send:
    gpg --delete-secret-and-public-key "Tin Ho (Tin Ho, email account)"
            will delete the entry form db, quote text string as output of --list-keys
    gpg -r "RECIPIENT UID" --armor --sign --encrypt < txtfile > cryptfile
            encrypting and signing a text file to be send as email, from tho@brio to tin@tokyo3
            (if no -r specified, will be prompted.  Will also prompt for passphrase of the
            private secret key for signing").
            pgp is fairly smart in matching strings for recipient,
            any unique thing listed in --list-keys can be used
            if using full name with space or symbols, enclose in quote.
    gpg --list-keys
            List keys of public signature imported into personal db.
    to send data w/o signing:
    gpg -e [RECIPIENT UID] < txtfile
    gpg -d < cryptfile
    to sign text message
    This only sign message, but does NOT encrypt:
    gpg --clearsign txtfile         #it will generate txtfile.asc:
    verify signature in signed file:
    gpg --verify txtfile.asc
    $ gpg --import keyname.asc
    $ gpg -v --verify [.asc file]
    gpg --import ovpn-security-key-2018.asc.txt		# import signer's certificate 
    gpg -v --verify openvpn-install-2.4.6-I602.exe.asc	# check against data file of same name sans .asc 


    od -c filename		# octal dump -C for ASCII or escaped char, to see \r \n of DOS textfile
    xxd -r		# hex to ascii, need input line number a la hexdump ?
    cd - (cd to previous dir)
    !$   (last param of previous command)  keyboard shortcut in bash is alt+.
    actually, don't think can just pipe to at.  so this won't send mail at later time... check danny's mail8r script... 
    make it work in n10 only is fine...
    use at for email reminder: mail -s 'subject' tho01 | at 10:00am {feb11}
    type message, ctrl-d to end
    note: dizzy support -s 'subject' , 
    grads, sdb1, does not support -s, just have subject: in message section
    n10 does not support -s option at all
    sh -x CMD	cmd is the regular way of executing a bourne shell script
    		invoked like this will show each command as they are carried out
    tail -f FILE monitor file as it grows
    sdiff  -s = do not print identical lines
    scp remote-host:remote-path local-host:local-path
    	secure copy b/w 2 comp on the network
    edit-assign vi = edit $HOME/.hpdrc-assign/assignment file
    rishen hours {project} {hrs} {supervisor_email} = report prj of past 2wk hours 
    assigning {uid} = edit assignment file of other users
    edit-rcs vi {filename} = edit assignment files of past employees, must be in the dir and use assignment as the filename 
    #/home/rvc-d1/mgutie01/rsi/envi/bin/envi to run demo ver of envi
    #/disk/73/agonza24/envi_3.0/bin/envi  for new version of envi on sdb1
    envi run as aliased command now
    gimp        image editor in Linux, avail for solaris
    sdtimage	image viewer in solaris CDE default setup, in /usr/dt/bin
    last        see who last logged on to the sys (a long log)
    acroread    launch acrobat reader for .pdf file
    xpsview     adobe ps viewer that comes with irix 6.5
    ghostview   view .ps file
    gs          view .ps file, command mode
    ghost       my script to launch gs in alex linux box
    ps2pdf      convert from ps to pdf file
    pushd .     push current dir into stack  (aliased to pu)
    popd        pop the pushed dir (ie change back to old dir; alias po)
    pushd \!*   jump b/w last pushed dir (aliased to xd)
    tar -zxvf file {-C dir}   untar and uncompress
    tar -zcvf file  *         tar and compress current dir, files still remain
    kill -9 -1 kill every single process i own w/in the server.
    fold -w80    wrap long line to max 80 column
    fmt -w{72}   wrap long line to less than 72, keeping word together
    mail user < filename
    mailx -s "subject" < filename (solaris)
          -v	# verbose, give SMTP session dump, good for debugging.
    to log out of sgi session remotely:
    /usr/bin/X11/endsession -f -display n10:0
    sendpage -q -p USERNAME "msg, myname"
    lpr  -P{PrinterName} {FileName}	 print file
    lp   -d {PrinterName} 
    lpq  -P{PrinterName}             show lpr queue 
    lprm -P{PrinterName} [JobNum]    mustang (delete) print job 
                                     if no JobNum, remove 1st submitted job
    # in solaris, will print two page per sheet, double sided.
    # mp is an ascii to ps prettyfier 
    # -l landscape, print two sheet per page
    # -o format ascii
    # mp is in /bin, /usr/bin, /usrlocal/bin , cs is /usr/openwin/bin/mp
    mp -l -o -s "subject/title" < infile | lp  -o nobanner -d chicago-duplex
    /etc/printers.conf contain list of installed printer 
    tar  -zxvf FILE [-C DIR ]      untar (and ungzip with z option)
    gzip -d    FILE.gz              ungzip .gz file
    uncompress File.Z               uncompress .Z file
    df -k              free hd space
    df -k .            free hd spcace on current hd only
    cat -vte			-vte = display all tab as ^I, eof marked by $   
    ypcat passwd	show /etc/passwd file from dns server
    yppasswd	change password on all system
    smbclient "\\hills\tho01"      connect to nt drive
    f45desm    form designer for oracle/unix in Motif
    r25desm    report designer, but does not seem to work
    /disk/1/app/oracle/product/732/bin/browser20em    oracle databrowser
    snapshot     screen/capture program for unix, diff version for sun and sgi
    run a program on remote computer w/o first logging in
    {host} {prog-name} -d {display-name} [param]
    eg:   n103 nxterm 
          grads netscape -d n10:0 &
    grep pattern 	-A 3 	# show the maching line and 2 more lines AFTER  it (no space works, ie -A3)
    		-B 3	# show the maching line and 2 more lines BEFORE it
    		-C 3    # show n lines before and after the match
    		-n3	# POSFIX, similar to -C 3, but add a line number column.  there must be NO spaces between n and the number!!
    xargs    	# grab a list of lines with single entry and turn it into a multi item single line list.  eg:
    		# ls -1  | xargs echo       			# will be equiv of echo *
    		# find /nfshome -name .rhosts | xargs chmod 600 
    /usr/bin/logger -p daemon.notice "test msg by tin"
    		append message to syslog, at level indicated by -p.

    [Doc URL:]
    (cc) Tin Ho. See main page for copyright info.