Saturday 31 December 2016



The Kahless_9 framework
c++ is already widely considered a very powerful language that provides very good performance and when combined with multi-threaded design can offer much more power and performance. The main issue with c++ however is that it is still consider by many to be a difficult language to use correctly relative to other languages like java and c# which are considered managed. 

Being a developer with capabilities in all these languages I can agree that c++ is more complicated than java or c# but my agreement comes with some reservations. I consider using c++ over java similar to someone using helicopter instead of a car to reach a particular destination. Most people consider a car easier to operate than a helicopter and rightfully so but making the complete trip with the car to reach one's destination might not be as easy and effective as doing it with a helicopter. Basically the car would make a good easy start but could very well end up not completing the journey or really battle to do so, whereas the helicopter will take more effort to get going but will most likely complete the journey efficiently.

This is similar in my view to using c++ over a managed language like java or c#. Please note that I am not trashing java or c# but am advocating for a greater use of c++ where it can be highly beneficial. Too often we hear reasons why c++ is not the best choice and this is evident by the much larger java and c# community as well as the belief over the last 10~15 years that c++ is going to be replaced by one of these other languages. Today, I will try to do the opposite by advocating some reasons why to choose c++ and dispel some of the reason people believe for rejecting it.

One reason I heard from a work colleague about why a particular component of a system was developed in vb instead of c++ was that c++ is too difficult. As mentioned before, I can agree that c++ is more difficult than other languages especially vb, but strangely here too, my car verses helicopter analogy is befitting. My colleague hastily used vb like a car to start a journey but found that journey eventually becoming too difficult to complete with a car and then I was called in with the c++ helicopter which needless to say completed the journey with overwhelming success.

The trick is to make c++ simple for yourself and this may be a tip that would help: do a basic c++ course that comes with good examples that relay the languages basics clearly. Then pick a c++ framework that fits he specific area of work you will be, or are in already. For front end development, Qt is probably one of the best ranked out there. For back end server type development, I can recommend Kahless_9.

Kahless_9 is a c++ framework consisting of multiple class libraries that exist for both windows and linux. The framework was recently released at site http://www.shankodev.com but has been collaboratively developed in various stages over the last 15 years by a group of specialist working in particular fields where high performance through multi-threading was critical to success.

The site also recently listed that it will be releasing Kahless_9 in dot NET versions for the visual studio 9 and 12 versions of the c++ framework. This means that if you know Kahless_9 c++ you will be able to easily apply the same concepts in c# or vb.net or even c++_cli. The reverse as well should be true in that a c# or vb.net developer that knows the Kahless_9 framework will also easily be able to adapt same concepts for c++.

Aside from the rich repository of threading abstractions available in Kahless_9, it also contains abstractions worth mentioning quickly that unifies certain structures and mechanisms in both linux and windows. Some examples of these are the date and time class KDate, the guid class and the file system management classes. Another striking feature of Kahless_9 is its ability to demonstrate multi-threaded action very quickly and easily in a frame that can easily be adapted for actual production use. This is partly accomplished by Kahless_9’s logging class which is thread-safe and logs the date and time down to millisecond as well as thread id of a particular log entry.

Consider the following sample that demonstrates using a logging class within two threads:

#include <conio.h>
#include "KLibIncUse.h"
#include "KLibThreadIncUse.h"



KLog Log("Logs\\Pxrs.log");



/************************************************************************************
**
** CPxrA
**
************************************************************************************/
class CPxrA : public KThreadProcessor
{
public:
    CPxrA() : m_nCntA(0 )
    {
       SetCycleSleepTime(3000);
    }


    ~CPxrA()
    {
       Stop();
    }

    void Process()
    {
       Log("m_nCntA = [%d]", m_nCntA);
       m_nCntA += 3;
    }



private:
    int m_nCntA;
};



/*************************************************************************************
**
** CPxrB
**
*************************************************************************************/
class CPxrB : public KThreadProcessor
{
public:
    CPxrB() : m_nCntB(2)
    {
       SetCycleSleepTime(2000);
    }


    ~CPxrB()
    {
       Stop();
    }

    void Process()
    {
       Log("m_nCntB = [%d]", m_nCntB);
       m_nCntB += 2;
    }


private:
    int m_nCntB;
};



/*************************************************************************************
**
** main
**
*************************************************************************************/
int main()
{
    Log.LogHeader();

    CPxrA pxrA;
    CPxrB pxrB;

    pxrA.Start();
    pxrB.Start();

    printf("Press any key to stop processing ... ");
    getch();

    printf("\nKey was pressed, now stopping pxrs\n");
    pxrA.Stop();
    pxrB.Stop();
    printf("pxrs has been stopped\n");

    return 0;
}


The following is typical output for the above program:
<2016-12-22 01:49:10.571> [00000330] *************************************************************
<2016-12-22 01:49:10.571> [00000330] *************************************************************
<2016-12-22 01:49:10.571> [00000330] ***
<2016-12-22 01:49:10.571> [00000330] *** Log Header:
<2016-12-22 01:49:10.571> [00000330] *** File: [multi_thread_log_tst]
<2016-12-22 01:49:10.571> [00000330] *** Version: [No Version]
<2016-12-22 01:49:10.571> [00000330] *** Name: [Free Demo Version]
<2016-12-22 01:49:10.571> [00000330] *** Company: [Free Demo Version]
<2016-12-22 01:49:10.571> [00000330] *** License: [Free Demo Version]
<2016-12-22 01:49:10.571> [00000330] ***
<2016-12-22 01:49:10.586> [00000330] *************************************************************
<2016-12-22 01:49:10.586> [00000330] *************************************************************


<2016-12-22 01:49:10.586> [00000930] m_nCntA = [0]
<2016-12-22 01:49:10.602> [00000adc] m_nCntB = [2]
<2016-12-22 01:49:12.614> [00000adc] m_nCntB = [4]
<2016-12-22 01:49:13.597> [00000930] m_nCntA = [3]
<2016-12-22 01:49:14.627> [00000adc] m_nCntB = [6]
<2016-12-22 01:49:16.608> [00000930] m_nCntA = [6]
<2016-12-22 01:49:16.639> [00000adc] m_nCntB = [8]
<2016-12-22 01:49:18.652> [00000adc] m_nCntB = [10]
<2016-12-22 01:49:19.619> [00000930] m_nCntA = [9]
<2016-12-22 01:49:20.664> [00000adc] m_nCntB = [12]
<2016-12-22 01:49:22.630> [00000930] m_nCntA = [12]
<2016-12-22 01:49:22.677> [00000adc] m_nCntB = [14]
<2016-12-22 01:49:24.689> [00000adc] m_nCntB = [16]
<2016-12-22 01:49:25.641> [00000930] m_nCntA = [15]
<2016-12-22 01:49:26.701> [00000adc] m_nCntB = [18]
<2016-12-22 01:49:28.651> [00000930] m_nCntA = [18]
<2016-12-22 01:49:28.714> [00000adc] m_nCntB = [20]
<2016-12-22 01:49:30.726> [00000adc] m_nCntB = [22]
<2016-12-22 01:49:31.662> [00000930] m_nCntA = [21]
<2016-12-22 01:49:32.739> [00000adc] m_nCntB = [24]
<2016-12-22 01:49:34.673> [00000930] m_nCntA = [24]
<2016-12-22 01:49:34.751> [00000adc] m_nCntB = [26]
<2016-12-22 01:49:36.763> [00000adc] m_nCntB = [28]
<2016-12-22 01:49:37.684> [00000930] m_nCntA = [27]
<2016-12-22 01:49:38.776> [00000adc] m_nCntB = [30]
<2016-12-22 01:49:40.695> [00000930] m_nCntA = [30]
<2016-12-22 01:49:40.788> [00000adc] m_nCntB = [32]


Explanation:
The sample consists of a main function and two classes [CPxrA] and [CPxrB]. Each of these classes inherit from [KThreadProcessor] which requires its derived classes to implement a [Process] method. Each of these [Process] methods run within their own KThreadProcessor’s thread instance cyclically. The sleep time in milliseconds between each cycle can be set via KThreadProcessor’s [SetCycleSleepTime]. In our sample [CPxrA]’s sleep cycle time is set to 3 seconds while [CPxrB]’s sleep time is set to 2 seconds. This means that [pxrA]’s Process method will execute every 3 seconds while [CPxrB]’s Process method will execute every 2 seconds.
Both pxrA and pxrB simply increment their local members [m_nCntA] and [m_nCntB] respectively after logging previous value. pxrA increments its value by 3 each cycle while pxrB increments its value by 2. The main function owns the pxrA and pxrB objects which it simply calls [Start] on. Thereafter the main function waits for us to press any key in order to call [Stop] on pxrA and pxrB which will stop the [Process] methods of these two objects from executing.
The log file produced demonstrates the date and time down to millisecond of each log as well as the thread id responsible for creating the log entry. From the log extract above we can see that pxrA’s thread id is [00000930] while pxrB’s thread id is [00000adc]. This is very important in multi-threaded development as it provides a visual detail of the how the various threads involved performed. Typically, such logs become your first and most valuable tool for solving multi-threaded related issues.


Another Approach
The results of the previous sample can also be attained by using the [KPxr] threading abstraction to create 2 individual threads within a single class that cyclically processes two separate methods of the class. This can be observed in the following code listing:


#include <conio.h>
#include "KLibIncUse.h"
#include "KLibThreadIncUse.h"


KLog Log("Logs\\Pxrs.log");



/***************************************************************************************
**
** CPxr
**
***************************************************************************************/
class CPxr
{
public:
    CPxr() : m_nCntA(0), m_nCntB(0)
    {
       m_pxrA.Activate(this, MyProcessA, 3000);
       m_pxrB.Activate(this, MyProcessB, 2000);
    }


    ~CPxr()
    {
       Stop();
    }

    void Start()
    {
       m_pxrA.Start();
       m_pxrB.Start();
    }


    void Stop()
    {
       m_pxrA.Stop();
       m_pxrB.Stop();
    }


    void MyProcessA()
    {
       Log("m_nCntA = [%d]", m_nCntA);
       m_nCntA+=3;
    }


    void MyProcessB()
    {
       Log("m_nCntB = [%d]", m_nCntB);
       m_nCntB+=2;
    }



private:
    int        m_nCntA;
    int        m_nCntB;
    KPxr<CPxr> m_pxrA;
    KPxr<CPxr> m_pxrB;
};



/**************************************************************************************
**
** main
**
***************************************************************************************/
int main()
{
    Log.LogHeader();

    CPxr pxr;
    pxr.Start();

    printf("Press any key to stop processing ... ");
    getch();

    printf("\nKey was pressed, now stopping pxrs\n");
    pxr.Stop();
    printf("pxrs has been stopped\n");

    return 0;
}


Explanation
Notice that unlike in the previous sample that had two classes namely [CPxrA] and [CPxrB], this sample only has one called [CPxr]. Class [CPxr] contains two methods called [MyProcessA] and [MyProcessB] which gets called cyclically by the two [KPxr] instances [m_pxrA] and [m_pxrB]. Within the constructor of [CPxr] we setup [m_pxrA] to process method [MyProcessA] and [m_pxrB] to process [MyProcessB] by calling their [Activate] methods to which we pass the method to be executed. The third parameter of the [Activate] method refers to the cycle sleep time for the particular processor. In the above sample we set [m_pxrA] to sleep for 3000 milliseconds while we instruct [m_pxrB] to sleep for 2000 milliseconds.


A Good Date and Time class
One thing a c++ developer tends to notice is that there are many date time structures to choose from, each with its own pro’s and conns. At times one may even find that what you use in windows might not be in linux or even the other way around. To this end, the framework provides a good date time class called [KDate], which can handle nearly all other date time structures and can represent a vaster time range than most other structures. This class also makes date time duration arithmetic very easy. Observe the following sample:

#include "KLibIncUse.h"

int main()
{
    KDate dtX(2016, 12, 21, 23, 37, 54); // 21-Dec-2016 23:37:54
    KDate dtY = dtX + 3*__1hour + 23*__1min + 6*__1sec; // 3 hrs, 23 mins & 6 secs after dtX
    KDate dtZ = dtY - 7.0; // 1 week before dtY

    cout << "dtX = [“ << dtX.ToString().sz() << “]” << endl;
    cout << "dtY = [“ << dtY.ToString().sz() << “]” << endl;
    cout << "dtZ = [“ << dtZ.ToString().sz() << “]” << endl;

    return 0;
}



The sample above will output the following:


dtX = [2016-12-21 23:37:54]
dtY = [2016-12-22 03:01:00]
dtZ = [2016-12-15 03:01:00]
Press any key to continue



File Management System classes
There are often cases within a system that one would need to perform some type of file management operations. Given that standard c++ does not cater for this it is fortunate that Kahless_9 comes equipped with such mechanisms to enabled uniformity in this regard for linux and windows. In essence the framework contains the classes [KFileIO], [KDirTraverser] and [KDirLister] to accomplish file management. Class [KFileIO] is handy for extracting the filename or file extension or returning the size of a file while the other two classes are handy for returning a directory listing and scanning through a directory tree and performing operations on the files and subdirectories found. Most ad-hoc development requiring a level of file management can easily be developed by using these three classes mentioned. The framework in particular demonstrates this by providing source code for utilities like [dcx], [fcx] and [find_files_btween_time] among several others. Let us review the source code for [find_files_between_time]:

class CConsoleMain
{
public:
    CConsoleMain();
    ~CConsoleMain();

    void ShowUsage();
    void ParseArgs(int argc, char * argv[]);
    void Process();

private:
    KStr m_strFileSpec;
    KDate m_dtStart;
    KDate m_dtEnd;
    bool m_bProcessSubDirs;


    static void _FuncDirTrv(const KStr & strDir, const _finddata_t & fi, void * pvInst);
};



void CConsoleMain::ParseArgs(int argc, char * argv[])
{
    …
    m_strFileSpec = argv[1];
    m_dtStart.FromString(argv[2]);
    m_dtEnd.FromString(argv[3]);
}


void CConsoleMain::Process()
{
    KDirTraverser dirTrv;
    dirTrv.SetFileSpec(m_strFileSpec);
    dirTrv.SetRecurtSubDirs(m_bProcessSubDirs);
    dirTrv.SetFileExecInfoFunc(_FuncDirTrv, this);
    dirTrv.Execute();
}



void CConsoleMain::_FuncDirTrv(const KStr & strDir, const _finddata_t & fi, void * pvInst)
{
    CConsoleMain * inst = (CConsoleMain *)pvInst;

    KDate dtCreate(fi.time_create);
    KDate dtModify(fi.time_write);

    bool bCreateIn = dtCreate>=inst->m_dtStart && dtCreate<=inst->m_dtEnd;
    bool bModifyIn = dtModify>=inst->m_dtStart && dtModify<=inst->m_dtEnd;

    if (bCreateIn || bModifyIn)
    {
       printf(" [%s] [%s] - %s\n", dtCreate.ToString().sz(), dtModify.ToString().sz(),
           KFileIO::Join(strDir, fi.name).sz());
    }
}
Explanation
The important methods to take note of for class [ConsoleMain] are [Process] and [_FuncDirTrv]. The [Process] method creates a [KDirTraverser] and sets the file specification to be searched for as well as the callback function [_FuncDirTrv] which it uses to process each file received by the directory traverser. The callback function [_FuncDirTrv] checks if the files create or modify time falls between the start end time specified and outputs to screen if it does. Even though this version of [KDirTraverser] accepting a callback function passing the [_finddata_t] file structure is windows specific, we can apply a similar solution for linux where we implement the callback function that passes the filename as a string. Thereafter our linux version of the callback function can obtain the files create and modify times by using [KFileIO].


Networking in Kahless_9
Development of network based application can be accomplished by using the frameworks socket abstractions for udp or tcp sockets. The socket abstractions consist of wrappers that encapsulate the functionality of udp and tcp sockets as well as a wrapper to simplify the handling of a socket address. Simply put, these abstractions will allow a developer to create a udp or tcp based socket application but without the need to know about or get into the messy syntax that is normally accompanied with socket based applications. The classes to take note of here are: [KSockUDP], [KSockTCP] and [KSockIp], and the examples available for download will demonstrate just how easy and neat these classes make socket application development. There are in addition other networking abstractions as well that further simplifies network based development and can provide performance that is not easily attained with normal socket level development. One of these classes is called [KTransmissionMgr] while the other is called [KTransmissionMgrHL]. Both these classes handle [KTransmission] objects with the only difference between them being that the [KTransmissionMgrHL] is intended for higher loads of networking traffic. The [KTransmissionMgr] basically uses a balanced tree (a 2-3 tree) in order to store its transmissions while the [KTransmissionMgrHL] uses a [K23TreeOrb] which does not suffer the same performance hit for a delete operation as does the normal 2-3 tree. Although the [KTranmissionMgr] suffers slightly when encountering higher loads where a higher rate of delete operations are required, it is by no means slow, the high load version just performs even better when encountering high loads. To demonstrate its use, consider the following code sample:

unsigned short usPort = 3773;

KTransmissionMgr mgr;
mgr.Activate(usPort);
mgr.SetFuncHndlTxmRx(FuncTxmRx, NULL);

…

void FuncTxmRx(KTransmissionRx * pTxmRx, void * pvParam)
{
   unsigned char * pBuffer = pTxmRx->GetBuffer();
    unsigned int uiSize = pTxmRx->GetBufferSize();
    // now process the receive buffer as you wish
}



For illustration, let us consider the above snippet to be the server based application while the next the client based. Technically however since both the client and server both contain [KTransmissionMgr] objects they are both equally capable of transmitting and receiving which actually makes their underlying architecture peer-to-peer as opposed to a strict client server model. The client code snippet is as follows:

unsigned short usPort = 3775;

KStr strText = “This is a text buffer to test KTranssmissionMgr”;

KTransmissionMgr mgr;
mgr.Activate(usPort);
mgr.Transmitt(“127.0.0.1”, 3773, strText.sz(), strText.GetLength());



Explanation
In both snippets listed above the [KTransmissionMgr] objects call their [Activate] methods which tell it which port to listen on. The [Activate] methods also allow other values for controlling the number of receiving sockets, number of transmitting sockets, number of threads per socket as well as the guid with which the [KTransmissionMgr] identifies itself.
In the server snippet above we set a callback function by calling [SetFuncHndlTxmRx] to handle incoming transmissions. The callback function [FuncTxmRx] has form: 

void FuncName(KTransmissionRx * pTxmRx, void * pvParam);

The client based snippet above calls its [Transmitt] method passing to it the ip address and port to where transmission should be sent as well as the buffer along with its size to transmit. We could have also setup callback functions to handle transmission success or failures from both tx and rx perspectives.


Synchronization
Synchronization has many forms each of which attempts to sequence action to a particular resource such that incoherent access is limited thus preventing race conditions or other threading concurrency related issues. Some of these synchronization constructs are heavy weight while some are light weight. A mutex can be considered a heavy weight synchronization construct since it applies a full lock across a process barrier and only releases lock when it is completely done with it thus causing complete blocking on all other waiting thread/process instances. A Critical Section on the other hand can be considered a lighter weight synchronization construct than a mutex as it only applies its lock between threads and not across process barriers but is heavier than a simple [KLock] that does not cater for a lock being applied more than once from same thread without being released first.
The best synchronization constructs to use would be the lightest ones where possible. Kahless_9 has some good lightweight synchronization constructs that if applied correctly within a solution would provide excellent overall performance. One such synchronization construct to consider is the [KSafeReadItem] which allows multiple readers to simultaneously read a resource while a single writer can update it cyclically without any locking/waiting. Consider the following sample:


struct SSomeStockTbls
{
    KDate dtUpdateTime;

    … // all other stock tbl values


};



SomeStockTblsMgr.h

class CSomeStockTblsMgr : public KThreadProcessor
{
public:
    CSomeStockTblsMgr();

    void Process();
    SSomeStockTbls * PtrTbls();


private:
    KSafeReadItem<SSomeStockTbls> m_sriStockTbls;


    void _LoadNewStockTbls(SSomeStockTbls & tbls);
};



SomeStockTblsMgr.cpp

CSomeStockTblsMgr::CSomeStockTblsMgr()
{
    SetCycleSleepTime(5*60*1000); //every 5 minutes
}


void CSomeStockTblsMgr::Process()
{
    SSomeStockTbls tbls;
    _LoadNewStockTbls(tbls);

    m_sriStockTbls.SetItem(tbls);
    m_sriStockTbls.SwitchCurrentPtr();
}


SSomeStockTbls * CSomeStockTblsMgr::PtrTbls() // concurrent multiple read thread access
{
    return m_sriStockTbls.GetCurrentPtr();
}


void CSomeStockTblsMgr::_LoadNewStockTbls(SSomeStockTbls & tbls)
{
    tbls.dtUpdateTime = KDate::Now();
    // complete rest of load from ftp, internet, ...
}


Explanation
We have some structure [SSomeStockTbls] that contains the update time among other records relevant to some business model. The class [CSomeStockTblsMgr] for managing updates and access to this structure stores a [KSafeReadItem] of it which is updated every five minutes in its [Process] method in its own thread inherited from [KThreadProcessor]. A [KSafeReadItem] stores 2 instances of the item type specified and keeps track of the current and noncurrent one. All reads/gets are done from the current item while the sets are all done to the noncurrent item. The [Process] method updates a local stack based instance of the structure called [tbls] by calling method [_LoadNewStockTbls]. Thereafter the [tbls] parameter is set to the safe read item [m_sriStockTbls] which will set the safe read items noncurrent instance of structure. After the call to [SwitchCurrentPtr] all new read threads will receive the newly updated structure while all other completing (trans phasing) read threads will be allowed to safely complete their already busy read operations on the old structure. Note that this model allows for multiple read requests from multiple threads concurrently without ever having to wait on the update of new structure values to complete. All read operations will be safe if they don’t exceed the cycle update time to complete.


Conclusion
Kahless_9 contains many other abstractions which also make coding easier and more secure but cannot realistically mention all of them within this blog. Even though the linux version of the framework does not contain the database and the mfc libraries, it can still be used to create very powerful applications, especially network based. Such applications can be fully developed and tested on windows before compiling and implementing on linux.



Posted by at No comments:
Subscribe to: Posts (Atom)