We Keep Coding

How do I apply the printers.py modification? (Linux OS)

I checked the core file because the process(c++ lang) running on Linux died, and the contents of the core file
[Corefile]
File "/usr/lib64/../share/gdb/python/libstdcxx/v6/printers.py", line 558, in to_string
return self.val['_M_dataplus']['_M_p'].lazy_string (length = len)
RuntimeError: Cannot access memory at address 0x3b444e45203b290f
I think that there was a problem with class StdStringPrinter at printers.py.
So I looked up a text that explained the problem I was looking for on this site , modified printers.py, and created a .gdbinit on my home path and wrote the content.
How to enable gdb pretty printing for C++ STL objects in Eclipse CDT?
Eclipse/CDT Pretty Print Errors
But this method is a little different from the one I'm looking for because it's done in Eclipse.
my gdb version is 7.6.1-94.el7
[printer.py]
class StdStringPrinter:
"Print a std::basic_string of some kind"
def __init__(self, typename, val):
self.val = val
def to_string(self):
# Make sure &string works, too.
type = self.val.type
if type.code == gdb.TYPE_CODE_REF:
type = type.target ()
sys.stdout.write("HelloWorld") // TEST Code
# Calculate the length of the string so that to_string returns
# the string according to length, not according to first null
# encountered.
ptr = self.val ['_M_dataplus']['_M_p']
realtype = type.unqualified ().strip_typedefs ()
reptype = gdb.lookup_type (str (realtype) + '::_Rep').pointer ()
header = ptr.cast(reptype) - 1
len = header.dereference ()['_M_length']
if hasattr(ptr, "lazy_string"):
return ptr.lazy_string (length = len)
return ptr.string (length = len)
def display_hint (self):
return 'string'
[.gdbinit]
python
import sys
sys.path.insert(0, '/home/Hello/gcc-4.8.2/python')
from libstdcxx.v6.printers import register_libstdcxx_printers
register_libstdcxx_printers (None)
end
My question is to modify printers.py, write gdbinit, and then re-compile the process to test whether it has been applied as modified.
How can I print my modified TEST code at Linux Terminal?

I think that there was a problem with class StdStringPrinter at printers.py
I think you are fundamentally confused, and your problem has nothing at all to do with printers.py.
You didn't show us your GDB session, but it appears that you have tried to print some variable of type std::string, and when you did so, GDB produced this error:
RuntimeError: Cannot access memory at address 0x3b444e45203b290f
What this error means is that GDB could not read value from memory location 0x3b444e45203b290f. On an x86_64 system, such a location indeed can not be readable, because that address does not have canonical form.
Conclusion: the pointer that you followed (likely a pointer to std::string in your program) does not actually point to std::string. "Fixing" the printers.py is not going to solve that problem.
This conclusion is corroborated by
the process(c++ lang) running on Linux died,
Finally, the pointer that you gave GDB to print: 0x3b444e45203b290f looks suspiciously like an ASCII string. Decoding it, we have: \xf); END;. So it's very likely that your program scribbled ); END; over a location where the pointer was supposed to be, and that you have a buffer overflow of some sort.
P.S.
My question is to modify printers.py, write gdbinit, and then re-compile the process to test whether it has been applied as modified.
This question also shows fundamental misunderstanding of how printers.py works. It has nothing to do with your program (it's loaded into GDB).
Recompiling anything (either your program or GDB) is not required. Simply restarting GDB should be all that's neccessary for it to pick up the new version of printers.py (not that that would fix anything).

C++ Implementation of the GDAL Algorithm GDALFillNodata()

Looking for some help with some C++ code to implement/call the GDALFillNoData() algorithm. I already have a working version using python and gdal, which is somewhat slow filling elevation DEMs (1.5GB). I'm curious if this is possible. I've written the code for a command line application and posted it here. File paths are hard coded at the moment. It builds (CodeBlocks 16.1/MinGW) and runs in but then crashes.
I'm not a C++ programmer, though I wish I were, but I'm trying to understand the language better. I'm moderately decent at python. I'm likely missing some thing basic here that's normal to C++.
There likely code that's been commented out through testing. So if something doesn't make sense that's why.
Here's the Code:
#include <iostream>
#include "gdal.h"
#include "gdal_priv.h"
#include "cpl_conv.h"
#include "gdal_alg.h"
int main()
{
GDALAllRegister();
//CPLPushErrorHandler(CPLQuietErrorHandler);
// Read/Write Files
const char *input = "D:/myIn.tif";
GDALDataset *pSrcDataset;
//GDALRasterBandH hMaskBand;
GDALRasterBand *poBand;
//CPLErr maskBand;
int maskFlags;
int noData;
double maxSearch = 10.0;
int maxInt = 1;
int nBlockXSize, nBlockYSize;
double adfGeoTransform[6];
//CPLErr eErr;
pSrcDataset = (GDALDataset*) GDALOpen(input, GA_Update);
CPLAssert( pSrcDataset != NULL );
poBand = pSrcDataset->GetRasterBand( 1 );
poBand->GetBlockSize( &nBlockXSize, &nBlockYSize );
printf( "Block=%dx%d Type=%s, ColorInterp=%s\n",
nBlockXSize, nBlockYSize,
GDALGetDataTypeName(poBand->GetRasterDataType()),
GDALGetColorInterpretationName(
poBand->GetColorInterpretation()) );
noData = pSrcDataset->GetRasterBand(1)->GetNoDataValue();
printf( "No Data Value = %i\n",noData );
printf( "Driver: %s/%s\n",
pSrcDataset->GetDriver()->GetDescription(),
pSrcDataset->GetDriver()->GetMetadataItem( GDAL_DMD_LONGNAME ) );
printf( "Size is %dx%dx%d\n",
pSrcDataset->GetRasterXSize(), pSrcDataset->GetRasterYSize(),
pSrcDataset->GetRasterCount() );
if( pSrcDataset->GetProjectionRef() != NULL )
printf( "Projection is `%s'\n", pSrcDataset->GetProjectionRef() );
if( pSrcDataset->GetGeoTransform( adfGeoTransform ) == CE_None )
printf( "Origin = (%.6f,%.6f)\n", adfGeoTransform[0],
adfGeoTransform[3] );
printf( "Pixel Size = (%.6f,%.6f)\n",adfGeoTransform[1],
adfGeoTransform[5] );
//maskBand = pSrcDataset->GetRasterBand(1)->GetMaskBand();
//hMaskBand = GDALGetMaskBand( maskBand );
//hMaskBand = pSrcDataset->GetRasterBand(1)->GetNoDataValue();
maskFlags = pSrcDataset->GetRasterBand(1)->GetMaskFlags();
printf ( "Mask Flags = %i\n", maskFlags );
printf ( "Processing image\n" );
// Ignore that this is on two lines here
GDALFillNodata(pSrcDataset, pSrcDataset->GetRasterBand(1)-
>GetMaskBand(), maxSearch, 0, maxInt, NULL, NULL, NULL);
//CPLAssert( eErr == CE_None);
GDALClose(pSrcDataset);
return 0;
}
Some errors that I'm getting when running this code (see image links). The process returns a 255 which is I think something that is unique to CodeBlocks?
Program Crashes
Process returns 255
Here is the python implementation. Pretty straight forward. Is None the same as NULL? Because one of the errors I got when using NULL as the hMaskBand (rasterfill.cpp)
#Run the gdal fill
ET = gdal.Open(infile, GA_Update)
ETband = ET.GetRasterBand(1)
result = gdal.FillNodata(targetBand = ETband, maskBand = None,
maxSearchDist = 500, smoothingIterations = 1)
print result # return 0
ET = None
Please let me know if you need more information. Knowing what little I know about C++ it's probably my build environment. :)
Thanks,
Heath

As a follow up for the comments above, the problem is the fact that the first argument given to the GDALFillNoData API is a GDALDataset* instead of a GDALRasterBand*.
But I've been struggling too much with GDAL to keep myself from giving a much explanatory answer.
The implementation of the GDAL lib is very C oriented, while it is indeed written using C++ features (e.g. classes).
So, you will find that in many of the available APIs the arguments are not referring to types defined in the library. Instead, they use typedef like the GDALRasterBandH, that are indeed alias to the void* type.
This has the nasty effect that, whatever type of argument you feed to the compiler, it won't complain, hence you will get a lot of errors at runtime if you make such (very common) mistakes.
Why they do this? I think it was a way to exploit PIMPL in order to avoid the compiler to work on several translation units, every time one of this classes was modified.
The real problem of this approach is that you lose all the benefits from static type checking, i.e. the compiler telling you when there is a type mismatch. As a side note, this is not a critics to the GDAL lib (thank God we have it!! And it is a huge and very useful project), it's just the way it is.
By the way, I'm currently working on a C++ opensource project, Rasterix, that is meant to be a user friendly GUI to GDAL tools for raster datasets. Do not take it as do it like me hint, as this app still needs proper testing, but there is a lot of GDAL APIs use cases there that may help if you need sort of examples on how to use them.

Dll causes Python to crash when using memset

I've been working on a project where I'm trying to use an old CodeBase library, written in C++, in Python. What I want is to use CodeBase to reindex a .dbf-file that has a .cdx-index. But currently, Python is crashing during runtime. A more detailed explanation will follow further down.
As for Python, I'm using ctypes to load the dll and then execute a function I added myself which should cause no problems, since it doesn't use a single line of code that CodeBase itself isn't using.
Python Code:
import ctypes
cb_interface = ctypes.CDLL("C4DLL.DLL")
cb_interface.reindex_file("C:\\temp\\list.dbf")
Here's the CodeBase function I added, but it requires some amount of knowledge that I can't provide right now, without blowing this question up quite a bit. If neccessary, I will provide further insight, as much as I can:
S4EXPORT int reindex_file(const char* file){
CODE4 S4PTR *code;
DATA4 S4PTR *data;
code4initLow(&code, 0, S4VERSION, sizeof(CODE4));
data = d4open(code, file);
d4reindex(data);
return 1;
}
According to my own debugging, my problem happens in code4initLow. Python crashes with a window saying "python.exe has stopped working", when the dll reaches the following line of code:
memset( (void *)c4, 0, sizeof( CODE4 ) ) ;
c4 here is the same object as code in the previous code-block.
Is there some problem with a dll trying to alter memory during runtime? Could it be a python problem that would go away if I were to create a .exe-file from my python script?
If someone could answer me these questions and/or provide a solution for my python-crashing-problem, I would greatly appreciate it.
And last but not least, this is my first question here. If I have accidently managed to violate a written or unwritten rule here, I apologize and promise to fix that as soon as possible.

First of all the pointer code doesn't point anywhere as it is uninitialized. Secondly you don't actually try to fill the structure, since you pass memset a pointer to the pointer.
What you should probably do is declare code as a normal structure instance (and not a pointer), and then use &code when passing it to d4open.

Like Joachim Pileborg saying the problem is, to pass a Nullpointer to code4initLow. ("Alternative") Solution is to allocate Memory for the struct CODE4 S4PTR *code = (CODE4*)malloc(sizeof(CODE4));, then pass the Pointer like code4initLow(code, 0, S4VERSION, sizeof(CODE4));.

Call functions in AutoIt DLL using Python ctypes

I want to call functions from an AutoIt dll, that I found at C:\Program Files (x86)\AutoIt3\AutoItX\AutoItX3.dll using Python. I know I could use win32com.client.Dispatch("AutoItX3.Control") but I can't install the application or register anything in the system.
So far, this is where I am:
from ctypes import *
path = r"C:\Program Files (x86)\AutoIt3\AutoItX\AutoItX3.dll"
autoit = windll.LoadLibrary(path)
Here are the methods that works:
autoit.AU3_WinMinimizeAll() # windows were successfully minimized.
autoit.AU3_Sleep(1000) # sleeps 1 sec.
Here is my problem, python is crashing when I call other methods like this one. I get python.exe has stopped working from windows...
autoit.AU3_WinGetHandle('Untitled - Notepad', '')
And some other methods are not crashing python but are just not working. This one doesn't close the window and return 0:
autoit.AU3_WinClose('Untitled - Notepad', '')
And this other one return 1 but the window is still minimized:
autoit.AU3_WinActivate('Untitled - Notepad', '')
I've tested the examples with with Dispatch("AutoItX3.Control") and everything is working like expected.
It seems like methods that should return something other than a string are crashing python. But still, others like WinClose are not even working...
Thank you in advance for your help!
EDIT:
These methods are now working when using unicode strings:
autoit.AU3_WinClose(u'Untitled - Notepad', u'')
autoit.AU3_WinActivate(u'Untitled - Notepad', u'')
And I found the prototype for AU3_WinGetHandle:
AU3_API void WINAPI AU3_WinGetHandle(const char szTitle,
/[in,defaultvalue("")]*/const char *szText, char *szRetText, int
nBufSize);
Now I can retrieve the return value using the following code!
from ctypes.wintypes import LPCWSTR
s = LPCWSTR(u'')
print AU3_WinGetHandle(u'Untitled - Notepad', u'', s, 100) # prints 1
print s.value # prints '000705E0'!
Thank you to those who helped me!

If you have the prototypes of the functions you're trying to call, then we can help you debug the calls without guessing. Or, more importantly, we won't have to help you debug the calls, because you can let ctypes do it for you.
See Specifying the required argument types in the docs.
For example, let's say the function looks like this (just a random guess!):
void AU3_WinClose(LPCWSTR name, LPCWSTR someotherthing);
You can do this:
autoit.AU3_WinClose.argtypes = (LPCWSTR, LPCWSTR)
autoit.AU3_WinClose.restype = None
If you do this, ctypes will try to convert your arguments to the specified types (LPWSTR, which is a pointer to wide char used for Windows UTF-16 strings) if it can, or raise an exception if it can't, and will not expect any return value.
If you don't do this, ctypes will try to guess the right things to convert your arguments to, possibly guessing wrong, and will try to interpret the non-existent return value as an int. So, it will usually crash until you managed to guess exactly what types to throw at it to make it guess the right types to pass to the function.

Will it work with unicode strings?
autoit.AU3_WinClose(u'Untitled - Notepad', u'')
autoit.AU3_WinActivate(u'Untitled - Notepad', u'')
Actually you might have to explicitly create unicode buffers, e.g.:
autoit.AU3_WinClose(create_unicode_buffer('Untitled - Notepad'), create_unicode_buffer(''))
Via some Googling, it looks like AU3_WinGetHandle takes 4 arguments, not 2. So you need to get that sorted out.

How can I call inlined machine code in Python on Linux?

I'm trying to call inlined machine code from pure Python code on Linux. To this end, I embed the code in a bytes literal
code = b"\x55\x89\xe5\x5d\xc3"
and then call mprotect() via ctypes to allow execution of the page containing the code. Finally, I try to use ctypes to call the code. Here is my full code:
#!/usr/bin/python3
from ctypes import *
# Initialise ctypes prototype for mprotect().
# According to the manpage:
# int mprotect(const void *addr, size_t len, int prot);
libc = CDLL("libc.so.6")
mprotect = libc.mprotect
mprotect.restype = c_int
mprotect.argtypes = [c_void_p, c_size_t, c_int]
# PROT_xxxx constants
# Output of gcc -E -dM -x c /usr/include/sys/mman.h | grep PROT_
# #define PROT_NONE 0x0
# #define PROT_READ 0x1
# #define PROT_WRITE 0x2
# #define PROT_EXEC 0x4
# #define PROT_GROWSDOWN 0x01000000
# #define PROT_GROWSUP 0x02000000
PROT_NONE = 0x0
PROT_READ = 0x1
PROT_WRITE = 0x2
PROT_EXEC = 0x4
# Machine code of an empty C function, generated with gcc
# Disassembly:
# 55 push %ebp
# 89 e5 mov %esp,%ebp
# 5d pop %ebp
# c3 ret
code = b"\x55\x89\xe5\x5d\xc3"
# Get the address of the code
addr = addressof(c_char_p(code))
# Get the start of the page containing the code and set the permissions
pagesize = 0x1000
pagestart = addr & ~(pagesize - 1)
if mprotect(pagestart, pagesize, PROT_READ|PROT_WRITE|PROT_EXEC):
raise RuntimeError("Failed to set permissions using mprotect()")
# Generate ctypes function object from code
functype = CFUNCTYPE(None)
f = functype(addr)
# Call the function
print("Calling f()")
f()
This code segfaults on the last line.
Why do I get a segfault? The mprotect() call signals success, so I should be permitted to execute code in the page.
Is there a way to fix the code? Can I actually call the machine code in pure Python and inside the current process?
(Some further remarks: I'm not really trying to achieve a goal -- I'm trying to understand how things work. I also tried to use 2*pagesize instead of pagesize in the mprotect() call to rule out the case that my 5 bytes of code fall on a page boundary -- which should be impossible anyway. I used Python 3.1.3 for testing. My machine is an 32-bit i386 box. I know one possible solution would be to create a ELF shared object from pure Python code and load it via ctypes, but that's not the answer I'm looking for :)
Edit: The following C version of the code is working fine:
#include <sys/mman.h>
char code[] = "\x55\x89\xe5\x5d\xc3";
const int pagesize = 0x1000;
int main()
{
mprotect((int)code & ~(pagesize - 1), pagesize,
PROT_READ|PROT_WRITE|PROT_EXEC);
((void(*)())code)();
}
Edit 2: I found the error in my code. The line
addr = addressof(c_char_p(code))
first creates a ctypes char* pointing to the beginning of the bytes instance code. addressof() applied to this pointer does not return the address this pointer is pointing to, but rather the address of the pointer itself.
The simplest way I managed to figure out to actually get the address of the beginning of the code is
addr = addressof(cast(c_char_p(code), POINTER(c_char)).contents)
Hints for a simpler solution would be appreciated :)
Fixing this line makes the above code "work" (meaning it does nothing instead of segfaulting...).

I did a quick debug on this and it turns out the pointer to the code is
not being correctly constructed, and somewhere internally ctypes is munging
things up before passing the function pointer to ffi_call() which invokes the
code.
Here is the line in ffi_call_unix64() (I'm on 64-bit) where the function pointer is saved
into %r11:
57 movq %r8, %r11 /* Save a copy of the target fn.
When I execute your code, here is the value loaded into %r11 just before
it attempts the call:
(gdb) x/5b $r11
0x7ffff7f186d0: -108 24 -122 0 0
Here is the fix to construct the pointer and call the function:
raw = b"\x55\x89\xe5\x5d\xc3"
code = create_string_buffer(raw)
addr = addressof(code)
Now when I run it I see the correct bytes at that address, and the function
executes fine:
(gdb) x/5b $r11
0x7ffff7f186d0: 0x55 0x89 0xe5 0x5d 0xc3

You might have to flush the instruction cache.
It is unclear (to me, anyway) whether mprotect() automatically does this.
[update]
Of course, had I read the documentation for cacheflush(), I would have seen that it only applies on MIPS (according to the man page).
Assuming this is x86, you might have to invoke the WBINVD (or CLFLUSH) instruction.
In general, self-modifying code needs to flush the i-cache, but as far as I can tell there is no remotely portable way to do so.

I'd suggest you try to get your code working in C first, then translate to ctypes. There's also something like CorePy if you just want to be able to execute assembly from Python.