Category Archives: Ubuntu

Detecting memory leaks of C++ application in Ubuntu

First, I tried Valgrind tool using the following command:

valgrind --tool=memcheck --leak-check=yes ./app

With some large QT application started for some short period I got the following output:

==7090== HEAP SUMMARY:
==7090==     in use at exit: 5,623,365 bytes in 36,268 blocks
==7090==   total heap usage: 32,454,680 allocs, 32,418,412 frees, 12,822,939,874 bytes allocated
................................
==7090== LEAK SUMMARY:
==7090== definitely lost: 20,163 bytes in 74 blocks
==7090== indirectly lost: 60,053 bytes in 1,273 blocks
==7090== possibly lost: 396,167 bytes in 2,169 blocks
==7090== still reachable: 4,834,822 bytes in 31,576 blocks
==7090== suppressed: 0 bytes in 0 blocks
==7090== Reachable blocks (those to which a pointer was found) are not shown.
==7090== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==7090==
==7090== For counts of detected and suppressed errors, rerun with: -v
==7090== Use --track-origins=yes to see where uninitialised values come from
==7090== ERROR SUMMARY: 20905 errors from 1583 contexts (suppressed: 15 from 2)

When I left this app running for the night (approximately 16 hours), I got the following summary:

==3816== HEAP SUMMARY:
==3816==     in use at exit: 7,746,701 bytes in 36,248 blocks
==3816==   total heap usage: 827,800,342 allocs, 827,764,094 frees, 105,404,761,516 bytes allocated
..................................
==3816== LEAK SUMMARY:
==3816== definitely lost: 19,835 bytes in 38 blocks
==3816== indirectly lost: 59,805 bytes in 1,237 blocks
==3816== possibly lost: 396,167 bytes in 2,169 blocks
==3816== still reachable: 6,958,734 bytes in 31,628 blocks
==3816== suppressed: 0 bytes in 0 blocks
==3816== Reachable blocks (those to which a pointer was found) are not shown.
==3816== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==3816==
==3816== For counts of detected and suppressed errors, rerun with: -v
==3816== Use --track-origins=yes to see where uninitialised values come from
==3816== ERROR SUMMARY: 13022 errors from 1574 contexts (suppressed: 10 from 2)

It means that the app leaks about 0.126 MB/hour ( (6958734 – 4834822) / 16 / 1024.0 / 1024.0) and totally for the night 2.02 MB, and probably it is not a leak, because the app has pointers to the allocated memory (it is reachable), but it does not clean exit.

To figure out how it works I built the following trivial program:

int main()
{
  int *a = new int[100];
}

with debug information and started the tool:

g++ -g -o ex ex.cpp
valgrind --tool=memcheck --leak-check=yes ./ex

In the output the tool shows line number 3 where memory leak is occurred:

==22042== Memcheck, a memory error detector
==22042== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==22042== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==22042== Command: ./ex
==22042==
==22042==
==22042== HEAP SUMMARY:
==22042==     in use at exit: 73,104 bytes in 2 blocks
==22042==   total heap usage: 2 allocs, 0 frees, 73,104 bytes allocated
==22042==
==22042== 400 bytes in 1 blocks are definitely lost in loss record 1 of 2
==22042==    at 0x4C2B800: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==22042==    by 0x40061B: main (ex.cpp:3)
==22042==
==22042== LEAK SUMMARY:
==22042==    definitely lost: 400 bytes in 1 blocks
==22042==    indirectly lost: 0 bytes in 0 blocks
==22042==      possibly lost: 0 bytes in 0 blocks
==22042==    still reachable: 72,704 bytes in 1 blocks
==22042==         suppressed: 0 bytes in 0 blocks
==22042== Reachable blocks (those to which a pointer was found) are not shown.
==22042== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==22042==
==22042== For counts of detected and suppressed errors, rerun with: -v
==22042== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

Links:

  1. How I can detect memory leaks of C++ application in Linux (Ubuntu OS)?
  2. How to check for memory leaks in a large scale c++ Linux application?
  3. Still Reachable Leak detected by Valgrind
  4. With Memcheck’s memory leak detector, what’s the difference between “definitely lost”, “indirectly lost”, “possibly lost”, “still reachable”, and “suppressed”?
  5. Valgrind: can possibly lost be treated as definitely lost?

Another alternative is calling mallinfo() function in C++ code. The following code gets something like total heap size of the process:

int usedmem = mallinfo().uordblks;

Using this function I wrote a QT widget that shows memory usage in application status bar:

#include <QLabel>
#include <malloc.h>

class MemoryStatusWidget : public QLabel
{
public:
    MemoryStatusWidget(QWidget *parent = nullptr);

public:
    void timerEvent(QTimerEvent *) override;
};

MemoryStatusWidget::MemoryStatusWidget(QWidget* parent)
    : QLabel(parent)

{
    startTimer(1000);
}

void MemoryStatusWidget::timerEvent(QTimerEvent *)
{
    int usedmem = mallinfo().uordblks;

    setText(tr("Memory Usage:%1 MB").arg(QString::number((double)usedmem / (1024 * 1024), 'f', 3)));
}

Detecting leaks on Windows

Just in case if you need to detect leaks on Windows (where Valgrind doesn’t work and will never work by design), give Deleaker a try. By the way it can be integrated with Qt Creator, see video on YouTube.

Compiling GDAL on Ubuntu Linux with SQLite and MySQL support

Fist install MySQL client libraries and check their location:

apt-get install libmysqlclient-dev
find / -name '*libmysqlclient*'

Then install SQLite:

sudo apt-get install sqlite3 libsqlite3-dev

Extract GDAL sources and configure supported modules (MySQL and SQLite are not compiled by default, so we need to specify them explicitly):

unzip gdal211.zip
cd gdal-2.1.1/
./configure --with-sqlite3 --with-mysql

this will output “MySQL support: yes” and “SQLite support: yes” along with other information.

(more…)

How to find files by permission on Ubuntu

Today I accidentally changed permissions of all the files in some folder /usr/lib/git-core with the following command:

chmod o-rx *

At first I thought that I have broken my system, but in few minutes I remembered that I have the same distribution on some other machine, I listed the same folder on this machine and used the following commands to restore original permissions:

chmod o+rx *
find -maxdepth 1 -type f -perm 645 -exec chmod o-x {} \;

645 means the files that initially were -rw-r–r— and became -rw-r–r-x. This two commands saved me a lot of time. After that I even count the number of files with all permission combinations:

find -maxdepth 1 -type f -perm 644 | wc -l
find -maxdepth 1 -type f -perm 755 | wc -l
find -maxdepth 1 -type f -perm -o=x | wc -l

Installing Git on Ubuntu 12.04 and enabling HTTP access with Nginx

Git is a good alternative for developers who need a version control supported on both Windows and Linux platforms. Below I provided basic steps for installing Git on Ubuntu 12.04 and enabling HTTP access to the repositories with Nginx web server.

Installing required packages

First, we need to install Nginx and Git packages:

apt-get install nginx git

by default Nginx processes will run as www-data, (check “user” keyword in /etc/nginx/nginx.conf configuration file). Git installation has not created any user yet.

(more…)

Installing Jabber Messaging Service on Ubuntu 12.04

Ejabberd (Jabber daemon written in Erlang programming language) can be easily installed on Ubuntu 12.04 Server (Precise Pangolin) with the following command:

apt-get install ejabberd

The only steps needed to make Ejabberd work after the installation are to specify admin user and hostname in /etc/ejabberd/ejabberd.cfg file:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Options which are set by Debconf and managed by ucf

%% Admin user
{acl, admin, {user, "admin", "developernote.com"}}.

%% Hostname
{hosts, ["developernote.com"]}.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

(more…)

Nginx 502 Bad Gateway error after updating Ubuntu 12.04

Today I updated my Ubuntu Server to 12.04.5 LTS (release 12.04, codename: precise), and got “502 Bad Gateway” on all my websites.

I checked Nginx log files and found that Nginx cannot open the socket created by PHP-FPM:

2014/09/11 19:01:03 [crit] 2741#0: *107 connect() to unix:/var/run/www-devnote.sock failed (13: Permission denied) while connecting to upstream, client: XXX.XX.X.XX, server: ~^(www\.)?(?<domain>.+)$, request: “GET /2014/04/using-a-wpf-control-in-a-mfc-application/ HTTP/1.1”, upstream: “fastcgi://unix:/var/run/www-devnote.sock:”, host: “developernote.com”

(more…)

How to encrypt MySQL database in Ubuntu 12.04 LTS

encrypt MySQL databaseProbably, the easiest way to encrypt MySQL database in Ubuntu is by using ecryptfs-utils. Install ecryptfs-utils:

apt-get install ecryptfs-utils

Mount /usr/local/encrypted directory and create mdf directory for MySQL data files (you will be prompted for passphrase and other options):

mkdir /usr/local/encrypted
mount -t ecryptfs /usr/local/encrypted /usr/local/encrypted
cd /usr/local/encrypted
mkdir mdf
chug.sh mysql mdf
chmod og-rwx mdf

(more…)

Ubuntu VPS Performance Monitoring

I run several PHP websites in Jooma and WordPress on Ubuntu VPS using Nginx as the web server with PHP-FPM for processing PHP. In most cases all the sites work fast enough, but sometimes (probably once a couple days) I got a message from Yandex (Russian search engine like Google) telling that one of my websites did not respond within 5 seconds. I set up Nginx slowlog and detected that sometimes HTTP requests are processed for more than 5 seconds:

(more…)

Monitoring CPU Steal Time with command line in Ubuntu

Bash command that writes CPU Steal Time along with other CPU usage statistics from top command header once in a second to MySQL database:

top -b -d 1 | ts ‘%Y-%m-%d %H:%M:%S’ | sed -rn “s/(.{19} )Cpu\(s\):\s+([0-9]+\.[0-9]+)%us,\s+([0-9]+\.[0-9]+)%sy,\s+([0-9]+\.[0-9]+)%ni,\s+([0-9]+\.[0-9]+)%id,\s+([0-9]+\.[0-9]+)%wa,\s+([0-9]+\.[0-9]+)%hi,\s+([0-9]+\.[0-9]+)%si,\s+([0-9]+\.[0-9]+)%st(.*)/INSERT INTO m_perf_stat (CDate, cpu_us,  cpu_sy,  cpu_ni, cpu_id,  cpu_wa,  cpu_hi,  cpu_si,  cpu_st) VALUES (‘\1′, \2, \3, \4, \5, \6, \7, \8, \9);/p” | mysql -u <username> -p<password> <database>

that’s the longest bash command I ever written in my life Улыбка

(more…)

Configuring Firewall to enable FTP, SSH and HTTP on Ubuntu

FirewallCheck your currently implemented firewall rules with the following command:

iptables -L

Examine the output. On a clean Ubuntu installation you will see an empty ruleset:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

(more…)