使用 psycopg2 将重新排列放入 postgres
只是想快速了解使用 psycopg2 将重新排列中的大量数据导入 postgres 的最佳方法(即最少编码)。我见过一些使用演员的东西,但我真的认为这会很困难,我可以在网上找到一些好的东西。
示例有 200 个变量的人口普查数据,这些变量在 rearray 中读取,许多列具有不同的数据类型。我只想浏览一下使用列名称和数据类型并输入到 postgres 中。
另外,如果有比 psycopy2 更好的东西,我愿意接受建议。
这是我发现的,尽管它进入 sqlight 并且方式错误。
elif driver=='sqlite3':
tups=cur.fetchall()
if len(tups)>0:
_cast = {types.BooleanType: numpy.bool,
types.IntType: numpy.int32,
types.LongType: numpy.int64,
types.FloatType: numpy.float64,
types.StringType: numpy.str,
types.UnicodeType: numpy.str}
try:
typelist=[_cast[type(tmp)] for tmp in tups[0]]
except KeyError:
raise Exception("Unknown datatype")
res = numpy.core.records.array(tups)
else:
return None
res=[res[tmp] for tmp in res.dtype.names]
except BaseException:
Just would like to get a quick idea on the best, meaning least coding, way to get lots of data in recarray into postgres using psycopg2. I have seen some stuff using cast but really I thought it would be strait forward and I could find something good on the web.
Example have census data with 200 variables read in recarray with different data types for many columns. I want to just sweep through using column names and data types and input into postgres.
Also if there is anything better than psycopy2 I am open to suggestions.
This is what I found although it goes into sqlight and the wrong way.
elif driver=='sqlite3':
tups=cur.fetchall()
if len(tups)>0:
_cast = {types.BooleanType: numpy.bool,
types.IntType: numpy.int32,
types.LongType: numpy.int64,
types.FloatType: numpy.float64,
types.StringType: numpy.str,
types.UnicodeType: numpy.str}
try:
typelist=[_cast[type(tmp)] for tmp in tups[0]]
except KeyError:
raise Exception("Unknown datatype")
res = numpy.core.records.array(tups)
else:
return None
res=[res[tmp] for tmp in res.dtype.names]
except BaseException:
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
Psycopg 可以使用新的适配器(当您想要将 Python 对象传递到 Postgres 时)或新的类型转换程序(当您想要从 Postgres 读取并获取 Python 对象时)进行扩展。
看起来你必须注册 一些现有的 numpy 类型适配器,例如:
通过这些注册,您可以直接使用 numpy 类型作为查询参数。
您可以在 邮件列表。
Psycopg can be extended using a new adapter (when you want to pass a Python object to Postgres) or a new typecaster (when you want to read from Postgres and get Python objects).
It looks like you have to register some of the existing adapters for the numpy types, for instance:
With these registrations you can use numpy types directly as query parameters.
You may get more details and help on the Mailing List.