• 欢迎访问开心洋葱网站,在线教程,推荐使用最新版火狐浏览器和Chrome浏览器访问本网站,欢迎加入开心洋葱 QQ群
  • 为方便开心洋葱网用户,开心洋葱官网已经开启复制功能!
  • 欢迎访问开心洋葱网站,手机也能访问哦~欢迎加入开心洋葱多维思维学习平台 QQ群
  • 如果您觉得本站非常有看点,那么赶紧使用Ctrl+D 收藏开心洋葱吧~~~~~~~~~~~~~!
  • 由于近期流量激增,小站的ECS没能经的起亲们的访问,本站依然没有盈利,如果各位看如果觉着文字不错,还请看官给小站打个赏~~~~~~~~~~~~~!

python如何使用mmap库映射文件到内存用法详解

python 水墨上仙 1549次浏览

python通过mmap库映射文件到内存用法详解

转自:http://blog.chinaunix.net/uid-20393955-id-1645587.html

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp示例使用的文本如下lorem.txt:

Lorem&nbspipsum&nbspdolor&nbspsit&nbspamet,&nbspconsectetuer&nbspadipiscing&nbspelit.&nbspDonec

egestas,&nbspenim&nbspet&nbspconsectetuer&nbspullamcorper,&nbsplectus&nbspligula&nbsprutrum&nbspleo,

a&nbspelementum&nbspelit&nbsptortor&nbspeu&nbspquam.&nbspDuis&nbsptincidunt&nbspnisi&nbsput&nbspante.&nbspNulla

facilisi.&nbspSed&nbsptristique&nbsperos&nbspeu&nbsplibero.&nbspPellentesque&nbspvel

arcu.&nbspVivamus&nbsppurus&nbsporci,&nbspiaculis&nbspac,&nbspsuscipit&nbspsit&nbspamet,&nbsppulvinar&nbspeu,

lacus.&nbspPraesent&nbspplacerat&nbsptortor&nbspsed&nbspnisl.&nbspNunc&nbspblandit&nbspdiam&nbspegestas

dui.&nbspPellentesque&nbsphabitant&nbspmorbi&nbsptristique&nbspsenectus&nbspet&nbspnetus&nbspet

malesuada&nbspfames&nbspac&nbspturpis&nbspegestas.&nbspAliquam&nbspviverra&nbspfringilla

leo.&nbspNulla&nbspfeugiat&nbspaugue&nbspeleifend&nbspnulla.&nbspVivamus&nbspmauris.&nbspVivamus&nbspsed

mauris&nbspin&nbspnibh&nbspplacerat&nbspegestas.&nbspSuspendisse&nbsppotenti.&nbspMauris

massa.&nbspUt&nbspeget&nbspvelit&nbspauctor&nbsptortor&nbspblandit&nbspsollicitudin.&nbspSuspendisse

imperdiet&nbspjusto.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp

数据读取:

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp使用mmap()函数可以创建内存映射文件。第一个参数是一个文件描述符,可以来自一个文件对象的fileno()方法或从os.open()。调用者要在调用mmap()前打开文件,并调用结束后关闭它。第二个参数以字节为单位,是映射文件的大小。如果值是0,映射整个文件。如果大于当前文件大小,则扩展这个文件。注意可选参数access:ACCESS_READ,ACCESS_WRITE,ACCESS_COPY。



import mmap
import contextlib
 
with open('lorem.txt', 'r') as f:
    with contextlib.closing(mmap.mmap(f.fileno(), 0,
                                      access=mmap.ACCESS_READ)
                            ) as m:
        print 'First 10 bytes via read :', m.read(10)
        print 'First 10 bytes via slice:', m[:10]
        print '2nd   10 bytes via read :', m.read(10)
 

&nbsp执行结果:



$ python mmap_read.py
First 10 bytes via read : Lorem ipsu
First 10 bytes via slice: Lorem ipsu
2nd 10 bytes via read : m dolor si
 

数据写入



import mmap
import shutil
import contextlib
 
# Copy the example file
shutil.copyfile('lorem.txt', 'lorem_copy.txt')
 
word = 'consectetuer'
reversed = word[::-1]
print 'Looking for    :', word
print 'Replacing with :', reversed
 
with open('lorem_copy.txt', 'r+') as f:
    with contextlib.closing(mmap.mmap(f.fileno(), 0)) as m:
        print 'Before:'
        print m.readline().rstrip()
        m.seek(0) # rewind
 
        loc = m.find(word)
        m[loc:loc+len(word)] = reversed
        m.flush()
 
        m.seek(0) # rewind
        print 'After :'
        print m.readline().rstrip()
 
        f.seek(0) # rewind
        print 'File  :'
        print f.readline().rstrip()

                

执行结果:



$ python mmap_write_slice.py
Looking for : consectetuer
Replacing with : reutetcesnoc
Before:
Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
After :
Lorem ipsum dolor sit amet, reutetcesnoc adipiscing elit. Donec
File :
Lorem ipsum dolor sit amet, reutetcesnoc adipiscing elit. Donec
 

使用ACCESS_COPY则不会改变实际存储的文件



import mmap
import shutil
import contextlib
 
# Copy the example file
shutil.copyfile('lorem.txt', 'lorem_copy.txt')
 
word = 'consectetuer'
reversed = word[::-1]
 
with open('lorem_copy.txt', 'r+') as f:
    with contextlib.closing(mmap.mmap(f.fileno(), 0,
                                      access=mmap.ACCESS_COPY)
                            ) as m:
        print 'Memory Before:'
        print m.readline().rstrip()
        print 'File Before  :'
        print f.readline().rstrip()
        print
 
        m.seek(0) # rewind
        loc = m.find(word)
        m[loc:loc+len(word)] = reversed
 
        m.seek(0) # rewind
        print 'Memory After :'
        print m.readline().rstrip()
 
        f.seek(0)
        print 'File After   :'
        print f.readline().rstrip()
 
                

执行结果:



$ python mmap_write_copy.py
Memory Before:
Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
File Before :
Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
Memory After :
Lorem ipsum dolor sit amet, reutetcesnoc adipiscing elit. Donec
File After :
Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
 

正则表达式

&nbsp&nbsp&nbsp&nbsp可以与正则表达式配合使用:



import mmap
import re
import contextlib
 
pattern = re.compile(r'(\.\W+)?([^.]?nulla[^.]*?\.)',
                     re.DOTALL | re.IGNORECASE | re.MULTILINE)
 
with open('lorem.txt', 'r') as f:
    with contextlib.closing(mmap.mmap(f.fileno(), 0,
                                      access=mmap.ACCESS_READ)
                            ) as m:
        for match in pattern.findall(m):
            print match[1].replace('\n', ' ')
                

执行结果:


$ python mmap_regex.py
Nulla facilisi.
Nulla feugiat augue eleifend nulla.
 

参考资料:mmap&nbsp(http://docs.python.org/lib/module-mmap.html)&nbsp


开心洋葱 , 版权所有丨如未注明 , 均为原创丨未经授权请勿修改 , 转载请注明python如何使用mmap库映射文件到内存用法详解
喜欢 (0)
加载中……