Reduce web server log size


How to exclude log messages based on IP and request type, so that access.log doesn’t take too much space too soon.
As a public facing streaming server, nginx server’s access.log grows fast for three reasons:
1. due to HLS streaming, most requests are about .ts segment files, which we don’t want to log.
2. some requests are from web crawlers such as Googlebot
3. some IPs request files that we have no idea what they are about, which need to be ignored.

Excluding certain messages from logs/access.log is configured in nginx.conf.
This statement below in nginx.conf says: log only if user defined variable $loggable is true.
access_log logs/access.log combined if=$loggable;
This statement can be in http{} or server{} or location{}.

To set $loggable, we have statements below in. e.g. the server{} section:
set $loggable 1;
if ( $ext_in_log = 0 ) {
set $loggable 0;
}
if ( $bot_in_log = 0 ) {
set $loggable 0;
}
if ( $ip_in_log = 0 ) {
set $loggable 0;
}
which says: if these three user defined variables, ext_in_log, bot_in_log, ip_in_log, are true, set $loggable to 0.

To set these three variables:
check files extension, .ts, to decide if the request is about a video segment.
map $request $ext_in_log {
~*(.*?)\.ts.* 0;
default 1;
}
check if the request is from web crawlers.
map $http_user_agent $bot_in_log {
~*Googlebot 0;
~*bingbot 0;
default 1;
}
check if the request is from certain IPs
geo $remote_addr $ip_in_log {
208.71.208.0/22 0;
172.105.77.67 0;
default 1;
}
Here is a sample nginx.conf file: http://riowing.net/p/nginx.conf

Advertisements

gdb for embedded device


Using gdb to debug this streaming device: MStar MSO9380 ARM Cortex-A53
On device, by telnet:
# wdog 0 # turn off watch dog
# gdbserver :5555 –attach $PROCESS_ID_MYAPP # part of debug build of rootfs

On host: Linux desktop:
$ arm-linux-gdb $FILE_FULLPATH_MYAPP # the file of PROCESS_ID_MYAPP with symbols

Inside (gdb):
set pagination off # reduce output
set print pretty on # print with indentation
set sysroot /mypath/rootfs #location of .so files, alias of solib-absolute-prefix
set solib-search-path /mypath/rootfs1:/mypath/rootfs2
target remote 10.15.22.22:5555 #connect to gdbserver
b My.cpp:57 #break at line 57
c #continue to run

breakpad in Uverse


I integrated breakpad into AT&T U-verse STB crash logging system starting 2015.
breakpad is google’s open-source multiplatform crash reporting system.
pros:
1. smaller dump file. The core dump is usually hundreds of MB, hundred times larger than breakpad dump, or mini dump.
2. its minidump file follows Windows MiniDump MDMP format, standard format generating call stack for all threads.
cons: in-process reporting, but fixed by Crashpad, out-of-process reporting

Integration:
1. Establish a way to link to breakpad library and run init:
This can be done without touching the main application binary file, which is named TV2MonoClient.
start application: LD_PRELOAD=/mnt/breakpadPlusMyCode.so TV2MonoClient
we hood up breakpad with TV2MonoClient in DllMain() of breakpadPlusMyCode.so
2. tell TV2MonoClient where to save the minidump upon crash in DllMain:
c++ code: new google_breakpad::ExceptionHandler(MinidumpDescriptor, NULL, dumpCallback, NULL, true, -1);
MinidumpDescriptor has the full file name of the minidump
dumpCallback is called when the dump has been created, where we can, e.g. upload the dump.
3. parse the minidump into plain text file
all the symbol files of TV2MonoClient and .so files are needed.
Parsing was done on a windows server, and the key tool is minidump_stackwalk.exe from google, like so:
minidump_stackwalk.exe minidumpfile .\appSymbolFiles >plainStackOutput.txt 2>err.txt
plainStackOutput.txt receives the call stacks.

Windows Manager


The role WM is not obvious in Window 10 and Gnome when mixed with desktop.
The GUI applicatin, guix in last post, cannot be moved/resized without WM.
dwm is used as WM here.
Install the binary: apt-get install dwm, $ dwm & to run it, through ssh.
Run  guix from ssh: $ guix Handstand1903.xbm
Run stterm by shortcut from vcxsvr: AltShiftEnter
Resize and drag guix:
on vcxsvr, dwm showing 1 2 … 9 tags.
Alt left mouse to drag, Alt right mouse to resize.
dwm draws guix’s tilte bar, which says “Titlebar not there by default” in guix.
guix doesn’t have to be started by WM, e.g. Alt-P, in order to be decorated by WM.

dwm