SSH, Tunnel, Port Forward

Ponder

  • PermitTunnel is disabled by default on openssh, it poses lots of serucrity problems for TCP ports, cuz sshd runs as root
  • Stunnel: create tunnel wrapper to secure otherwise non encrypted TCP traffic
  • sshuttle: mount NFS from ssh connection. NFS server should only allow priviledged ports to mounts to mitigate this problem.
    sShuttle is amazing, but it also creates lots of security complications
    Ref: SSHUTTLE - A VPN for the Lazy
  • ssh agent forwarding. with git not allowing password, this is increasingly a necessity. But there are security risks. SSH_AUTH_SOCK can be read by root, and hijack the session to impersonate the user running the agent foward. There are mitigations, see
    SSH Agent Explained
    In sshd.conf, AllowAgentForward is independent of AllowTcpForwarding (and X11Forwarding).
  • SSH with SOCKS5

    Host socks5-browser HostName my_office_machine IdentityFile /home/.ssh/my_ssh_key DynamicForward 3125 # LogLevel DEBUG # ^^ DEBUG will print lot of message on the terminal that ran the ssh cmd. ssh socks-browser # to open connection (ssh will use the HostNHame clause) on firefox, set the network settings to use a socks5 proxy using the dynamic port listed above

    Scripting SSH Login

    Example Perl/Expect script to ssh to a host (eg iDRAC interface) with interactive password and issue command interactively: sshExpect.pl This is obviously insecure since the script has clear text password in it. it would be better to have ssh key pair created so that the client can login without password so this scripting doesn't have to use clear text password.

    Legacy cipher

    to login to old netapp needing 3des-cbc (which is now disabled cuz it is subject to attack), try one of these:
    ssh -c 3des-cbc old-netapp-filer
    ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 old-netapp-filer

    SSH Hostkey

    deleting / removing host key that has been changed (eg host moved, os reinstalled, etc)
    
    ssh-keygen -R greyhound
    ssh-keygen -R greyhound.fqdn.com
    ssh-keygen -R 123.3.32.87
    
    
    "Dealing" with it :)
    only if you absolutely trust the host and don't need to verify its identity. eg in an HPC cluster env :-D
    
    ssh -o UserKnownHostsFile=/dev/null
    
    alias ssh='ssh -o StrictHostKeyChecking=no'
    
    


    client config

    edit ~/.ssh/config for client side directive to ssh cli client
    eg use specific key for account@specific server, eg git client config.
    ref https://docs.gitlab.com/ee/ssh/
    Host gitlab.com
      Preferredauthentications publickey
      IdentityFile ~/.ssh/id_rsa
    Host *
    	ServerAliveInterval 240
    	ServerAliveCountMax 4
    

    SSH Tunnel

    Tunneling  with ssh using ssh port forward
    
    ssh -R 2023:raven:23 codadmin-1 
    
    -R report port 2023
    tunnel it to raven port 23
    remote machine is coadmin-1
    login, and the session will be in place
    
    in this eg, on coadmin-1, do telnet localhost 2023 will have a ssh tunnerl to raven, then connect to port 23 (telnet) on raven box.  This allow login to raven from coadmin-1 using telnet, but no clear text will be send across the network!
    
    connection last .... (after logoug of ssh, still reamins... at least for a while...)
    
    
    eg2:
    mis06$  ssh -R 30000:devapp01:21 -l tin raven.unixville.com
    then, in raven's prompt, a 'telnet localhost 30000' will connect to the ftp server on devapp01 (port 21)
    read as: Remote port 30000   send to devapp01:21  the rest is just login to the remote machine.
    once the ssh session is killed, the channel disapears (though just simple exit with preconnected channel will be forked to background).
    
         -R port:host:hostport 	remotesvr	#R = remote
         -L port:localhost:hostport remotesvr	#L = local
    
    floride# ssh -L 443:localhost:22 root@grads
    	localhost:443 will be LISTEN state, and tunnel to grads:22, which need root to work.
    	(if use tho01, ssh will complain that it is admin prohibited)
    	floride is linux laptop at home, 
    voivod$  ssh -L 2022:localhost:443 tho01@sn -p 23
    	voivod is at work, where it can connect to home linux on port 23, 
    	home router register IP as sn.is-a-geek.com, routing to floride
    	localhost:2022 tunnel to floride:443, it will ask for password thru ssh on port 23.
    
    Note that this double tunneling will connect, but need root access for port < 1024 :(
    each machine only open port in 127.0.0.1:* so cant remotely connect.
    
    
    
    
    --- 
    
    X11 forwarding...
    
    deepnds on configuration of both server and client to fwd X11
    ssh has an -X option to force it (but what if server refuse, then i guess it wont work then).
    
    ONce ssh into the remote machine, even thru several machines, will forward the X11 port correctly.  it set the display to some virtual display on the local machine, on :10.0 (port 6010) or something.  all the work will be behind the scene tthru the many virtual xserver that ssh setup on the way
    
    
    if using vnc and want to wrap that traffic inside ssh, 
    ssh -L 5911:localhost:5901  remoteVncSvr 
       then in a separate window on the localmachine, 
    vncviewer localhost:11
       this connect to local port 5911, wrap by ssh, then into the remote vncserver port 5901, ie :1
    
    ssh -Y -o ServerAliveInterval=300 -o ServerAliveCountMax=2 -L 5951:localhost:5911 tin@bofh  # 2400x1300 # **this vnc to localhost:51  which tunnel to bofh:11**
    
    ssh port forward tunnel chaining 
    
    ssh chaining
    ssh -L 5907:localhost:5907 tin@gt1
    ssh -L 5907:localhost:5907 tin@a1
    vncserver -geometry 1280x800  :7
    vncviewer localhost:7
    
    ---
    
    copy file from remote place, though only 1 file at a time
    
    ssh tho01@grads "cat file" > file.txt
    ssh tho01@grads " cat binfile | uuencode - " | uudecode -p > binfile
    cat binfile | uuencode - | ssh +C mis02 " uudecode -p > binfile"
    
    (had addded - for uuencode to expect stdin (wont work w/o cat though), and -p for uudecode to output to stdout (or else it dump into a file named '-').  grads cshrc noisy output will affect this working!)
    
    -p is in solaris only, presummably linux use diff option to output to stdout
    
    
    tin-u10#  cd /mnt/na_data_vol1/depot/jumpstart/OS/OS.fromJUMPSTART
    ssh root@10.215.2.16 "(cd /jumpstart/OS.local; tar cf -  sol_9_sparc sol_8_202_ia) | uuencode - " | uudecode -p | tar xf -
    
    #-jumpstart# cd /jumpstart/OS.local
    #- tar cf - * | uuencode - | ssh +C 10.215.2.42
    
    -----
    
    ssh or scp to server w/o providing password.
    
    on client, do ssh-keygen
    get content of $HOME/.ssh/identity.pub
    drop it in server $HOME/.ssh/authorized_keys
    
    
    
    linux:
    ssh-keygen -t rsa1
    ssh-keygen -t rsa -b 4096 
    ssh-keygen -t dsa
    ssh-keygen -t ed25519
    cd .ssh
    cat identity.pub id_dsa.pub >> authorized_keys
    
    chmod 600 authorized_keys
    
    key format of dsa acceptable for sshd v2
    rsa with at least 2048 bit recommended for gitlab.
    ed25519 is best in 2020.
    
    ## RSA with 1024+ bit recommended since 2002
    ## RSA with 3072+ bit recommended 2016 NSA
    ## AES-128 ... ?
    ## AES-256 is better than rsa-2048
    
    linux ssh is very pick nicky now.
    If parent dir is world writable, it will not accept the private key and insist on password :(
    It doesn't display error, it just keep on assking for password even when everything seems fine!!!
    
    perms should be:
    [ussf:.ssh] $ ls -la
    total 88
    drwx------   2 tinh ack1  4096 Jan 23 14:44 .
    drwxr-xr-x  46 tinh ack1 36864 Jan 23 16:42 ..
    -rw-r-----   1 tinh ack1   949 Nov  2 23:48 authorized_keys
    -rw-------   1 tinh ack1   672 Nov  2 23:47 id_dsa
    -rw-r-----   1 tinh ack1   610 Nov  2 23:47 id_dsa.pub
    -rw-------   1 tinh ack1   535 Nov  2 23:46 identity
    -rw-r-----   1 tinh ack1   339 Nov  2 23:46 identity.pub
    -rw-r--r--   1 tinh ack1 24689 Jan 23 14:31 known_hosts
    [ussf:.ssh] $
    
    
    to change / rekey  password of existing keypair:
    
    ssh-keygen -p -f ~/.ssh/id_rsa
    
    EXPECT that existing key stored in remote allowed_hosts can remain functional. 
    man page implies this is not generating a brand new key pair, just changing the passphrase needed to access it.
    
    
    ssh-agent
    
    ssh-agent reduces typing in password when using privatekey that is password protected.
    Since password remains in memory, and has socket in a path, 
    it is still possible to be abused by root...
    
    ssh-agent $SHELL	# agent terminates when SHELL exits
    
    ssh-agent 		# run in background forever, 
    			# but need to export SSH_AUTH_SOCK var manually in other shell session before agent work...
    			# eg:
    SSH_AUTH_SOCK=/tmp/ssh-xAcBqJ0gHFAd/agent.1130; export SSH_AUTH_SOCK; # wsl/Ubuntu
    export SSH_AUTH_SOCK=/var/folders/wy/jztf8lmn08v4kry0vsxzskpw0000gn/T//ssh-oQzD1UBJO3AO/agent.12475 # rhel?
    export SSH_AGENT_PID=12476	# not really needed
    
    
    ssh-add			# by default, add private keys in ~/.ssh/ to agent's memory.
    
    ssh somehost		# private key, those with passphrase, will be automatically inoked by ssh-agent
    
    
    
    
    # newer version of ssh will communicate automagically with ssh-agent ?
    # just that mac os x 10.12.5 has old ssh client that don't do this?
    # or maybe ssh-agent was disabled on mac?
    # 
    # actually: 
    # GNOME starts ssh agent on startup, and all shell start with proper env var set for SSH_AUTH_SOCK
    # ssh-add may still need to be run on the first shell session to let ssh-agent load the private key into its memory.
    # ref
    # https://unix.stackexchange.com/questions/72552/whats-the-purpose-of-ssh-agent
    
    

    sshfs

    sshfs  tin@dtn.brc.berkeley.edu:/global/scratch/users/tin   ~/mnt/brc-gs
    sshfs  -o ServerAliveInterval=300 -o ServerAliveCountMax=2  tin@dtn.brc.berkeley.edu:/global/scratch/users/tin  ~/mnt/brc-gs
    
    works even in wsl :D
    
    if using vscode, there maybe some nuance, see
    https://askubuntu.com/questions/1385124/unable-to-open-visual-studio-code-when-inside-sshfs-mounted-drive-ubuntu-wsl
    

    OpenSSL

    certs, and stuff

    test/eyeball certs

    
    # human readable parse of a domain cert:
    openssl x509 -text -noout -in domain.crt 
    
    # check expiration date of a .pem file:
    openssl x509 -enddate -noout -in greyhound.pem
    
    # cli connect to a server (eg internal docker registry) and display cert it is using:
    openssl s_client -showcerts -connect registry:443
    
    # wget --no-cert-check ...
    
    

    generating self-signed certs

    
    openssl req -new -x509 -sha256 -newkey rsa:2048 -nodes -sha256 \
        -keyout domain.key -out domain.crt  -days 4650 \
        -subj "/C=US/ST=CA/L=Berkeley/O=LBNL/OU=GRETA/CN=registry.greta.local:443" \
        -config self-signed.conf
    
    # -sha256 is for domain.crt domain.key format ?
    # omitting it generate pem format?
    
    self-signed conf
    (haven't figure out how to do multiple DNS name SAN req in cli, so have to use this config file)
    the req is like "main" then it tell what other sections to parse
    
    [ req ]
    x509_extensions = x509_ext
    distinguished_name = dn
    
    [dn]
    C = US
    ST = California
    L = Berkeley
    O = Lawrence Berkeley National Laboratory
    OU = GRETA
    CN = registry.greta.local:443
    
    
    [x509_ext]
    keyUsage = keyEncipherment, dataEncipherment
    extendedKeyUsage = serverAuth
    subjectAltName = @alt_names
    
    [ alt_names ]
    DNS.1   = registry.greta.local
    DNS.2   = registry:443
    DNS.3   = registry
    DNS.4   = registry.greta.local:443
    
    
    other commands to try if above dont work
    # citrix method https://support.citrix.com/article/CTX135602
    openssl req -x509 -nodes -days 3285 -newkey rsa:2048 -keyout domain.key -out domain.crt -config san.conf -extensions 'v3_req'
    
    # human readable parse of a domain cert:
    openssl x509 -text -noout -in domain.crt 
    
    cp -pi      domain.crt /clusterfs/gretadev/data/registry_certs
    chmod 0400  domain.key                                           # secure private key
    mv -i       domain.key /clusterfs/gretadev/data/registry_certs   # secure private key
    chown 771              /clusterfs/gretadev/data/registry_certs/domain.key
    # docker registry seems to be internally using uid 771 in the container (v2.0)
    
    
    
    # docker registry cert for internally hosting ref: https://docs.docker.com/registry/insecure/#use-self-signed-certificates
    openssl req \
      -newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key \
      -addext "subjectAltName = DNS:myregistry.domain.com" \
      -x509 -days 365 -out certs/domain.crt
    
    # git only take .pem certs, convert .crt to .pem as:
    openssl x509 -in greyhound.crt -out greyhound.pem -outform PEM
    
    
    
    
    Cert Request (for CA signing, eg lets encrypt)
    
    
    
    # signing a new cert request with a SAN (Subject Alternate Name)
    openssl req -new -out greyhound_lbl_gov.csr -newkey rsa:2048 -nodes -sha256 -keyout greyhound_lbl_gov.key.temp -config san.cnf
    
    
    # cat san.conf 
    [ req ]
    default_bits       = 2048
    distinguished_name = req_distinguished_name
    req_extensions     = v3_req
    prompt = no
    [ req_distinguished_name ]
    C = US
    ST = California
    L = Berkeley
    O = Lawrence Berkeley National Laboratory
    OU = GRETA
    CN = registry:443
    [ v3_req ]
    keyUsage = keyEncipherment, dataEncipherment
    extendedKeyUsage = serverAuth
    subjectAltName = @alt_names
    [alt_names]
    DNS.1   = registry
    DNS.2   = registry:443
    DNS.3   = registry.gretadev
    DNS.4   = registry.gretadev:443
    DNS.5   = registry.gretadev.lbl.gov
    DNS.6   = registry.gretadev.lbl.gov:443
    
    certs for grafana container w/ ssl
    
    # generate ssl certs, both domain.crt/cert.pem (public)  and domain.key/key.pem (private) will generated by the openssl cmd
    # openssl need to be installed , it is an interactive cmd
    
    # domain.crt is likely a diff format than cert.pem
    # but grafana took it just fine
    
    
    mkdir CERTS_PRIV
    
    #xx openssl req -newkey rsa:4096 -nodes -sha256 -keyout CERTS_PRIV/domain.key  -x509 -days 365 -out CERTS_PRIV/domain.crt
    #   openssl req -newkey rsa:4096 -nodes -sha256 -keyout CERTS_PRIV/key.pem     -x509 -days 865 -out CERTS_PRIV/cert.pem
    
    
    
    Other ssl cert testing commands
    
    openssl s_client -connect freeipa.greta.gov:443 -showcerts
    
    Ref:
  • EasyRSA self-signed certs (used with openVPN)
  • OpenSC ssh tunnel
  • ssh -o StrictHostKeyChecking=no example.com

    [Doc URL: http://tin6150.github.io/psg/ssh.html ]
    Last Updated: 2017-06-08
    (cc) Tin Ho. See main page for copyright info.


    hoti1
    sn5050
    psg101 sn50 tin6150