Windows API Hooking — Malware Analysis
This blog present a methodology while creating new tool to perform Windows API calls interception and recording it into a text file through using Inline Hooking. It discusses the problem and the benefits from Malware Analysis perspective.
Introduction
Windows Operating System has an API to interact with it as any other system. Windows Operating System has two modes: User Mode and Kernel Mode. all of user’s programs run on the user mode then the Operating System (OS) takes the call from the user mode to kernel mode as per the Windows OS architecture to be performed on the low level and the results to be reverted back to the user’s program in the user mode. for more information about windows modes, please check the below LINK
Any Windows program that makes activities on the system, is making windows API calls at some point of time to interact with operating system. these activities can be File System activities, Network activities, Information gathering activities about the system, or any other activities.
For example, File System activities can be reading, modifying, or deleting files.
What is Malware?
Malware is a normal computer program, but with malicious intents according to the Malware type. The malicious intents of any malware can be programmed to be performed once it infect the victim or it can be happened after calling the attacker to get the malicious instructions to be performed. at the end, all of these activities requires the malware author to use the Windows API function to perform his activities.
Malware Analysis
During the Malware analysis, the analyst tries to understand the behavior of the suspicious file to decide if it is malicious or not. the analysis done over two phases: Static Analysis and Dynamic Analysis. Finally the analyst is to generate an analysis report about the suspicious sample, and create a detection rule (Such as YARA rule) if it was malware.
Static Analysis
one of the main points to be checked during the static analysis, is to check the Imported Function, so that the analyst can find what windows API functions, the malware will use during its run time.
For example, the below fig.2, shows the network related functions, which the malware is importing from WS2_32, which indicate the the malware most probably will have a network activities.
The above information is important to the analyst to be used during the analysis of this malware, So with some cluse like this, the analyst will have a way to trace the malware behavior.
By end of the day, Static Analysis is not enough to conduct a full analysis for a suspicious file, as it has many limitations that can be stopper for the analyst, for example, if the suspicious file is packed and will be unpacked dynamically in the memory during the runtime, so the imported functions will not be shown in the static analysis. other obfuscation techniques are used by malware authors to hide the malicious activities of the malware and make the analysis phase is harder. for more information about packing and obfuscation, check the below links: Link1, Link2
Dynamic Analysis
Dynamic Analysis is to allow the malware to run in an isolated environment and continue analyzing it by inspecting its activities live. By using the Dynamic Analysis we can by somehow defeat the limitations of the Static Analysis.
The Dynamic Analysis phase can be mixed between Automation and Manual. for the automation part an automated Sandbox can be used to perform an analysis for the suspicious file and generate a report, however most of modern open source and commercial sandbox tries to perform both static and dynamic analysis, but the import part now, is the dynamic analysis.
The Sandbox will run the suspicious file with recording all its activities then apply some detection rules on the results to decide if it is malicious or not.
The Malware analyst can build an isolated machine with sperate internet connection to run the suspicious file inside it and with help of some tools, can detect and inspect the results running the sample.
Is it easy to that simplicity? for sure, it is not.
The Problem..
Malware authors over the years, have developed a lot techniques to detect if the malware is running inside Sandbox or even on a debugging machine using different ways. If the malware detected that it’s running inside a sandbox, in most cases it will terminate itself and maybe remove itself to make the analysis harder.
Anti-debugging techniques is the names of the used techniques by malware authors to detect and evade the sandboxes. As this is not the main target of this blog post, so for more information about it, please check this LINK for getting knowledge about the most common anti-debugging techniques.
Finally the results of the automated sandboxes will not be reliable during analyzing such evasive type of malware.
Regarding to performing this operation in semi-automated way with help of some tools such as: Microsoft SysInternals, Regshot, and others, it will take time and efforts from the malware analyst.
The Solution
Now, it is obvious that we need to run the malware in an isolated environment with taking into consideration the need for removing any clues of being an analysis machine, as this is not the main target of the blog post to describe the recommendations for this setup, so i can just mention some points and leave the other to another blog post.
It is highly recommended to perform the below on that machine:
- Give it some realistic specs, such as: more than 2GB RAM and more than 1 CPU core.
- Install a normal user applications, such as web browsers, Microsoft office, pdf reader, create some files.
- DONT install any related tools to the hypervisors such as: VM Tools.
- Give it separate internet connection.
- I do prefer not to install debuggers on this machine.
- some GOOGLE search can give more hints.
after performing all of the above points, you can clone this machine and use it as a GUEST machine while you are building sandbox such as Cuckoo
I have started to write a tool to act as Windows API proxy, to intercept the windows API calls and log all of the passed arguments and finally write it into a text file.
The Targeted Windows API calls
During the dynamic analysis of the malware, there are some important windows API calls to be identified
- Process
2. File System
3. Registry
For each category i listed sample of the important functions in the above screenshots.
API Hooking
Both endpoint security solution agent and malware perform Windows API calls hooking. Endpoint security solution agent monitors the made calls by any program on the system to detect any malicious activities and on the other side the malware author can use the hooking for stealing sensitive information or even modify the response of any API call.
Inline API Hooking
Simply it redirects the flow of the execution from function to another function (Hooking Code) and then getting the execution flow back to the original function (Hooked Function).
Normal execution flow
Hooked execution flow
Steps of Inline Hooking
- The first n bytes of the target function (the function we want to hook) are copied to a memory location often referred to as a “trampoline”.
- n bytes of the target function are overwritten by a relative JMP instruction to pass execution to our user hook code.
- After executing our “hook” code, execution return to the original function by passing execution to our trampoline which contains the bytes that we overwrote in the original function.
- After executing the overwritten bytes, a relative JMP takes the execution back to the original function at an offset after our added JMP command to prevent recursion.
There are many types of API hooking, but i have used the Inline Hooking to overcome the issue when the malware performs a dynamic linking through using LoadLibrary and GetProcessAddress function, which happens can bypass the Import Address Table Hooking. However this issue may be solved if we hooked GetProcessAddress function also, but i decided to go with Inline Hooking.
I have used Microsoft Detours Framework for performing Inline Hooking to facilitate intercepting and logging the windows API calls, as it overwrites the first few instructions of the target function, by a jump into the user-defined detour function. The trampoline function first stores the lost bytes, which was overwritten in the targeted function by the JMP instruction then JMP back to the targeted function with the remaining instructions, which are not overwritten. finally the target function returns to the detour function. considering the ability of Detours to perform pre-processing and post-processing before going to trampoline function and after getting back.
This type of hooking called Inline Hooking, for more details about different types of hooking, please check this Reference.
It’s highly recommend to read its Wiki and also this reference for more understanding about it before continue reading.
The two important required points to use Detours is to have a pointer to the targeted function and a detour function.
For example, let’s explain it by the below snippet of code.
First, getting a pointer for the targeted function: CreateProcess, and it can be done by two method. Both works fine and no need here to get into more in this point.
- First Method
static BOOL(WINAPI* realCreateProcess)(LPCWSTR lpszImageName, LPWSTR lpszCmdLine, LPSECURITY_ATTRIBUTES lpsaProcess, LPSECURITY_ATTRIBUTES lpsaThread, BOOL fInheritHandles, DWORD fdwCreate, LPVOID lpvEnvironment, LPCWSTR lpszCurDir, LPSTARTUPINFOW lpsiStartInfo, LPPROCESS_INFORMATION lppiProcInfo) = CreateProcess;2. Second Method
HMODULE Krnel32DLL = GetModuleHandle(L"kernel32.dll");
realCreateProcess = (PtrRealNtCreateFile)GetProcAddress(Krnel32DLL, "CreateProcess");Then let’s create a detour function as below:
BOOL WINAPI HookedCreateProcess(LPCWSTR lpszImageName, LPWSTR lpszCmdLine, LPSECURITY_ATTRIBUTES lpsaProcess, LPSECURITY_ATTRIBUTES lpsaThread, BOOL fInheritHandles, DWORD fdwCreate, LPVOID lpvEnvironment, LPCWSTR lpszCurDir, LPSTARTUPINFOW lpsiStartInfo, LPPROCESS_INFORMATION lppiProcInfo)
{
// Write Code Here!!
return realCreateProcess(lpszImageName, lpszCmdLine, lpsaProcess, lpsaThread, fInheritHandles, fdwCreate, lpvEnvironment, lpszCurDir, lpsiStartInfo, lppiProcInfo);
}Now we have the pointer to the targeted function and the detour function and both has to have same number and type of arguments.
Let’s have a look to the real implemented code inside the already mentioned detour function: CreateProcess
BOOL WINAPI HookedCreateProcess(LPCWSTR lpszImageName, LPWSTR lpszCmdLine, LPSECURITY_ATTRIBUTES lpsaProcess, LPSECURITY_ATTRIBUTES lpsaThread, BOOL fInheritHandles, DWORD fdwCreate, LPVOID lpvEnvironment, LPCWSTR lpszCurDir, LPSTARTUPINFOW lpsiStartInfo, LPPROCESS_INFORMATION lppiProcInfo)
{
logCall += L"\n[+] CreateProcess : ";
logCall += lpszImageName;
logCall += L" : ";
logCall += lpszCmdLine;
logCall += L" : ";
logCall += lpszCurDir;
logCall += L" : ";
logCall += L" : ";
return realCreateProcess(lpszImageName, lpszCmdLine, lpsaProcess, lpsaThread, fInheritHandles, fdwCreate, lpvEnvironment, lpszCurDir, lpsiStartInfo, lppiProcInfo);So inside the detour function, we log the values of some arguments and append it to logCall, which the buffer of logs to be written to the disk later.
Now inside the DllMain() function, will start by calling DetourIsHelperProcess, as my tool as it should run on both x86 and x64 systems, so this function will help by checking if the current process is a helper process or a target process to avoid intercepting calls of helper process.
Inside the case of DLL_PROCESS_ATTACH will start opening a stream into the logging file to write the time and process ID and after that will call the InstallHook() function to perform the hooking and in the case of DLL_PROCESS_DETACH will call the UninstallHook() function to stop the hooking and finally write the logs.
BOOL APIENTRY DllMain(HMODULE hModule, DWORD ul_reason_for_call, LPVOID lpReserved)
{
if (DetourIsHelperProcess()) { return TRUE; }
HANDLE currProc = GetCurrentProcess();
switch (ul_reason_for_call)
{
case DLL_PROCESS_ATTACH:
{
auto start = std::chrono::system_clock::now();
std::time_t end_time = std::chrono::system_clock::to_time_t(start);
ofstream myfile;
myfile.open("C:\\Windows\\Temp\\API_Calls_Monitor.txt");
myfile << "[+] DLLMain Entry.\n" << "[+] API Calls Monitor Started at: " << std::ctime(&end_time) << "[+] process ID: " << GetCurrentProcessId() << endl;
myfile.close();
DisableThreadLibraryCalls(hModule);
InstallHook();
break;
}
case DLL_THREAD_ATTACH:
{
break;
}
case DLL_THREAD_DETACH:
{
break;
}
case DLL_PROCESS_DETACH:
{
UninstallHook();
writeLog(logCall, true);
break;
}
}
return TRUE;
}Before getting in to InstallHook() function, the below function has to be used for using detours:
// This function is used to avoid repetition of interception as we will use DetourCreateProcessWithDllEx() and we dont want to intercept the calls of the helper process
DetourRestoreAfterWith();
// This function to mark the detour transaction
DetourTransactionBegin()
// This function to enlist the included threads in the transaction to update the instruction pointers when the transaction is comitted
DetourUpdateThread();
// This function to enable the hooking
DetourAttach( _Inout_ PVOID * targetedFunctionPointer, _In_ PVOID DetourFunctionPointer);
// This function to commit the transaction and start the hooking
DetourTransactionCommit();the same is applied inside UninstallHook() function except that we will replace DetourAttach() by DetourDetach() to stop the hooking.
According to all of the above, the InstallHook() function should be as the below, if we will intercept just CreateProcess, which we have mentioned in the targeted pointer section
void InstallHook()
{
DetourRestoreAfterWith(); // avoid repetition
DetourTransactionBegin(); // start hook
DetourUpdateThread(GetCurrentThread());
// Process APIs Functions
DetourAttach(&(PVOID&)realCreateProcess, HookedCreateProcess);
DetourTransactionCommit(); // commit hook
}void UninstallHook()
{
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());
// Process APIs Functions
DetourDetach(&(PVOID&)realCreateProcess, HookedCreateProcess);
DetourTransactionCommit();
}Now we have created a DLL to be injected to any suspicious process to intercept and log the targeted windows API calls then save the logs into text file.
There are many ways to inject the created DLL into the suspicious process even by using ProcessHacker, but …
The main issue here is that we will miss the API calls between the process start and the DLL injection happens, So the solution here is by using DetourCreateProcessWithDllEx or DetourCreateProcessWithDlls , which will create a new process and load the DLL into it, so we will not miss any API call.
This function also helps to support both x86 and x64 targeted programs on single system, but it will require to have both x86 and x64 compiled version of the DLL which will be injected inside the same directory with same name, but each one will be ended by its type: file32.dll and file64.dll.
I have used DetourCreateProcessWithDlls and considered other point which is the allowed time for the suspicious PE to run, then to be terminated and logs to be written to the disk, as the tool should buffer the logs in memory then write it to the disk after terminating the process. This behavior to avoid having too much I/O operations during monitoring the suspicious process, also to avoid any code complexity by handling the case of the having file writing operation by the suspicious process and same for writing the logs.
In the below code snippet, just preparing the required variables and getting the required inputs from the user
#include <iostream>
#undef _UNICODE
#include <cstdio>
#include <windows.h>
#include <detours.h>
#include <string>
#include <iostream>
#pragma warning(disable:4996)
using namespace std;
int main()
{
STARTUPINFO si;
PROCESS_INFORMATION pi;
// Value Initializion
ZeroMemory(&si, sizeof(STARTUPINFO));
ZeroMemory(&pi, sizeof(PROCESS_INFORMATION));
si.cb = sizeof(STARTUPINFO);
string targetedProcess; //Holds the path of the suspicious PE.
string dllToInject; //Holds the path of the DLL, which will be injected.
int timeToRun = 0; // The allowed running time of the process before terminating it.
//Getting the PATH of the under testing PE.
cout << "[*] Enter the Path of the targeted PE: ";
getline(cin, targetedProcess);
//Getting the PATH of the DLL
cout << "[*] Enter the Path of the DLL \" (ex: C:\\Windows\\Temp\\APICallsMonitor.dll) ";
getline(cin, dllToInject);
//Getting the allowed time to run the PE.
cout << "[*] Enter the time to run the targeted process under monitoring (in Min): ";
cin>>timeToRun;
//Just converting the type of var to be suitable to passed to the function
std::wstring stemp = std::wstring(targetedProcess.begin(), targetedProcess.end());
LPCWSTR sw = stemp.c_str();Now let’s call the function to create the process with loading the DLL into it
// Creatintg the process for the targeted PE and injecting the DLL
if (!DetourCreateProcessWithDllEx(sw,
NULL, NULL, NULL, TRUE,
CREATE_DEFAULT_ERROR_MODE| CREATE_SUSPENDED,
NULL, NULL, &si, &pi,
dllToInject.c_str(), NULL)) {
printf("[+] DLL Injection Failed!\n");
}
else {
cout << "[*] Created Process with Process ID: " << pi.dwProcessId << endl;
printf("[+] DLL Injection Successed!\n");
// Resume the excution of the created process of suspicious PE.
ResumeThread(pi.hThread);
printf("[+] Leave The process running for %d min\n", timeToRun);
// Just printing the remaining time to kill the process in min.
for (size_t i = 0; i < timeToRun; )
{
printf("\t [+] %d Min is remainaing\n", (timeToRun - i));
Sleep((++i) * 1000 * 60);
}
printf("\n[+] Terminating the Process.");
char fullCMD[30];
sprintf_s(fullCMD, "%s%d", "taskkill /pid ", pi.dwProcessId);
int retval = ::system(fullCMD);
WaitForSingleObject(pi.hProcess, INFINITE);
CloseHandle(pi.hProcess);
}So once the process created and the DLL is injected successfully, the tool waits till the running time to be finished then the process will be terminated.
Testing Results
This tool was tested on Windows 10 x64 and windows 7 x86 and worked fine.
Results
- APICallsMonitor_x64.dll : The API Calls Monitor in x86.
- DLLDetoursInjector_x64.exe : The Hooker which will create the process in Suspended mode then inject the DLL and let it runs for specific time then the process will be terminated and logs to be written to the logging file.
- The target is a malware sample, which i was analyzing at that time: 62b467a43acaba783b64205eec144f3f9b90717201a30a557bcb90dd3bb2b9eb
The above two screenshots were for the malware while it was running live.
However this malware closed itself before the time ends, but the logs also was writting to the disk.
The below screenshot is from the log file as sample of the output of the tool at that time.
Conclusion
However the tool is not finished and it needs some enhancements, but it was the purpose of the blog post to publish it. The main objective was to present the walkthrough of building something that may enhance the process of the malware analysis or having a quick view to the malware behavior outside the automated sandboxes.
Have you liked the post?
