Re: [pcre-dev] Searching a big file with PCRE

Top Page
Delete this message
Author: Maël Hörz
Date:  
To: Craig Silverstein
CC: pcre-dev
Subject: Re: [pcre-dev] Searching a big file with PCRE
> } Unfortunately this will not work (on a 32-bit system) since my files
> } could be larger than 2GB, see 3 GB and more.
>
> I dont' know what O/S you're using, but maybe you could use mmap64?

I'm using Windows, but it has equivalent memory file mapping functions
to mmap64. The issue is that the address space on a 32-bit system can be
too small to map the entire file (if larger than 2GB or 4GB). But if I
understood correctly pcre_exec needs to have the complete file contents
available in one chunk to search it correctly. So memory mapping doesn't
help me as I can only have a part of the file mapped to memory and
processed by pcre_exec.

Any other ideas? Is there perhaps a way that PCRE could tell me, "I need
to process the next chunk since I hit the buffer boundary"?

Regards, Maël.