我在炼金术中挣扎的东西。假设我有两个模型:
TextLocator
----------------
id (primary key)
docid
span_start
span_end
UNIQUE constraint docid, span_start, span_end
tags = relationship('Tag', back_populates='text_locator')
Tag
----------------
id
locator_id (Foreign Key text_locator.id)
tag_name
text_locator = relationship('TextLocator', back_populates='tags')
如果我导入一个包含一百万行的csv,并且每行是:
docid, span_start, span_end, tag_name
在SQLA中,将其插入标记表的最有效方法是什么
我面临的问题是找到需要创建和查询的TextLocator
如果我一次做一排,我可以做如下:
text_locator_instance = (
session.query(TextLocator)
.filter(
and_(docid == <docid>. span_start == <span_start, span_end == <span_end>)).first()
if not text_locator_instance:
text_locator_instance = TextLocator(docid=<docid>, span_start=<span_start>, span_end=<span_end>)
tag_instance = Tag(tag_name=<tag_name>)
tag_instance.text_locator = text_locator_instance
我将如何在100万行的规模上做到这一点
如果它是纯SQL,我会将100万个CSV插入一个临时表,连接到TextLocator以获取ID,在连接失败的地方插入TextLocator,然后通过连接插入带有所有ID的标记
目前没有回答
相关问题 更多 >
编程相关推荐