Transaction / RawSequel - 以及测试的乐趣
这两者都适用于“生产”,后者仅适用于测试。现在我已经在生产和测试中得到了一些东西,我想了解为什么我必须采用整个游标路线而不是 Django 路线。我相信问题与交易有关,但我并不乐观,因为我晚上 8:30 坐在这里,这让我很烦恼。
这与此问题相关我以为我已经有了答案(和理解),但可惜我没有。我的测试是 A/B,其中 A 在 Django 外部注入,B 与已知的 A 进行比较。提供的答案解决了我的部分问题,但当我添加更多测试时,问题不断重新出现。
我深入研究并假设 RawQuery 没有提交事务,但没有金额 transaction.commit 似乎解决了这个问题。我还从中删除了 django.testing.TestCase 并直接进行单元测试。我想我已经尝试了每种组合,但我对 SQL 或事务支持不是很精通,现在我想知道为什么一种有效而另一种无效...
如果有人在这里有任何见解,我真的会感谢!
更新 2 已修改并清理,但仍然失败..
# BUG: https://code.djangoproject.com/ticket/12768
# - Requirement for pmProp.* - This (in-part) forced me to shift to raw.
sqlraw = """ SELECT
pmProp.propid as propid_id,
pmProp.owner as owner,
pmProp.ownertype as ownertype,
pmProp.behavior as behavior,
pmProp.value as value_id,
pmPropDef.id as propdef_id,
pmPropDef.name as name,
pmPropDef.datatype as datatype,
pmPropDef.choicetype as choicetype,
pmPropDef.definition as definition_id,
pmPropDef.ptrig as prop_trigger,
pmPropDef.units as units,
IFNULL(pmPropShort.str, pmPropLong.str) as str_value FROM pmProp
INNER JOIN pmPropDef ON pmProp.propid=pmPropDef.id AND pmPropDef.name=%s
LEFT JOIN pmPropShort ON sid=pmProp.value
LEFT JOIN pmPropLong ON lid=-pmProp.value
WHERE pmProp.ownertype=%s AND pmProp.owner=%s AND pmPropDef.id=pmProp.propid
"""
if explicit:
sqlraw += " AND pmProp.behavior='explicit'"
# TRY ONE - DOES NOT WORK FOR TESTING..
# This will NOT work for testing - It simply doesn't get the value
# when repeatedly inserting from pm and checking the value.
#
# Note if you use this you must update the sqlraw to include pmProp.* bug..
#
#try:
# result = list(Property.objects.raw(sqlraw, [property, owner, self.id]))[0]
# result.value = self.coerce_datatype(result.str_val, result.datatype)
#except IndexError:
# result = None
# END TRY ONE
# Try TWO: THIS WORKS for both
cursor = connections['catalog'].cursor()
cursor.execute(sqlraw, [property, owner, self.id])
row = cursor.fetchone()
transaction.commit_unless_managed(using='catalog')
if row:
field_map = "propid_id owner ownertype behavior value_id propdef_id "
field_map += "name datatype choicetype definition_id prop_trigger "
field_map += "units str_value"
field_map = field_map.split()
class PropVal(object): pass
result = PropVal()
result.__dict__=dict(zip(field_map, row))
result.value = self.coerce_datatype(result.str_value, result.datatype)
try:
log.info("%s %s=%s %s" % (property.capitalize(), result.name,
result.value, result.units))
except UnicodeDecodeError: pass
else:
result = None
# END TRY Two
更新
这是一个示例 A/B 测试。
from django.db import connection, transaction
from unittest import TestCase
#from django.test import TestCase, TransactionTestCase
from apps.pmCatalog.utility.ICMPM.pm import Pm
from apps.pmCatalog.models import Property, Site, Project, Variant, Library, LibraryType, Release
import settings
import datetime
import logging
log = logging.getLogger(__name__)
class PropertyTests(TestCase):
def test_add_property_value(self):
"""Test the ability to add a property and retrieve a property"""
prop_types = [("string", "Funny Business"), ("integer", 1234), ("real", 12.34) ]
pm = Pm(mysql_db='test_bugs')
tree = pm.add_release_tree()
for prop_type, pmvalue in prop_types:
# Add a property definition for a branch (like a project)
pmproperty = "%s_%s_basic" % (tree[0].name, prop_type)
pm.add_property_definition(pmproperty, prop_type=prop_type)
pm.add_propval(pmproperty, value=pmvalue, project=tree[0].name)
#Project.objects.update()
project = Project.objects.get(name=pmproject.name)
property = project.get_property(pmproperty)
#When using the first one this ALWAYS returned None!
self.assertEqual(str(pmvalue), property.str_value)
self.assertEqual(pmvalue, property.value)
谢谢!
Both of these work in "production" The later only works in testing. Now that I've gotten something to work in both production and testing I'd like to understand why I had to go the whole cursors route rather than the Django route. I believe the problem has to do with transactions but I'm not positive and as I sit here at 8:30pm it's bugging me.
This is related to this question where I thought I had my answer (and an understanding) but alas I didn't. My testing is an A/B where A is injected outside of Django and B compares against the known A. The answer that was provided solved part of my problem but when I added in more tests the problem kept resurfacing.
I dug in and assumed it was the RawQuery was not committing the transaction but no amount transaction.commit seemed to fix it. I also removed the django.testing.TestCase
from it and went straight unittest. I think I've tried every combination but I'm not very proficient in SQL or Transactional support and now I'm left wondering why one works and one doesn't...
If anyone has any insight here I would really appreciate it!
Update 2 Revised and cleaned up but still failing..
# BUG: https://code.djangoproject.com/ticket/12768
# - Requirement for pmProp.* - This (in-part) forced me to shift to raw.
sqlraw = """ SELECT
pmProp.propid as propid_id,
pmProp.owner as owner,
pmProp.ownertype as ownertype,
pmProp.behavior as behavior,
pmProp.value as value_id,
pmPropDef.id as propdef_id,
pmPropDef.name as name,
pmPropDef.datatype as datatype,
pmPropDef.choicetype as choicetype,
pmPropDef.definition as definition_id,
pmPropDef.ptrig as prop_trigger,
pmPropDef.units as units,
IFNULL(pmPropShort.str, pmPropLong.str) as str_value FROM pmProp
INNER JOIN pmPropDef ON pmProp.propid=pmPropDef.id AND pmPropDef.name=%s
LEFT JOIN pmPropShort ON sid=pmProp.value
LEFT JOIN pmPropLong ON lid=-pmProp.value
WHERE pmProp.ownertype=%s AND pmProp.owner=%s AND pmPropDef.id=pmProp.propid
"""
if explicit:
sqlraw += " AND pmProp.behavior='explicit'"
# TRY ONE - DOES NOT WORK FOR TESTING..
# This will NOT work for testing - It simply doesn't get the value
# when repeatedly inserting from pm and checking the value.
#
# Note if you use this you must update the sqlraw to include pmProp.* bug..
#
#try:
# result = list(Property.objects.raw(sqlraw, [property, owner, self.id]))[0]
# result.value = self.coerce_datatype(result.str_val, result.datatype)
#except IndexError:
# result = None
# END TRY ONE
# Try TWO: THIS WORKS for both
cursor = connections['catalog'].cursor()
cursor.execute(sqlraw, [property, owner, self.id])
row = cursor.fetchone()
transaction.commit_unless_managed(using='catalog')
if row:
field_map = "propid_id owner ownertype behavior value_id propdef_id "
field_map += "name datatype choicetype definition_id prop_trigger "
field_map += "units str_value"
field_map = field_map.split()
class PropVal(object): pass
result = PropVal()
result.__dict__=dict(zip(field_map, row))
result.value = self.coerce_datatype(result.str_value, result.datatype)
try:
log.info("%s %s=%s %s" % (property.capitalize(), result.name,
result.value, result.units))
except UnicodeDecodeError: pass
else:
result = None
# END TRY Two
Update
Here is a sample A/B test.
from django.db import connection, transaction
from unittest import TestCase
#from django.test import TestCase, TransactionTestCase
from apps.pmCatalog.utility.ICMPM.pm import Pm
from apps.pmCatalog.models import Property, Site, Project, Variant, Library, LibraryType, Release
import settings
import datetime
import logging
log = logging.getLogger(__name__)
class PropertyTests(TestCase):
def test_add_property_value(self):
"""Test the ability to add a property and retrieve a property"""
prop_types = [("string", "Funny Business"), ("integer", 1234), ("real", 12.34) ]
pm = Pm(mysql_db='test_bugs')
tree = pm.add_release_tree()
for prop_type, pmvalue in prop_types:
# Add a property definition for a branch (like a project)
pmproperty = "%s_%s_basic" % (tree[0].name, prop_type)
pm.add_property_definition(pmproperty, prop_type=prop_type)
pm.add_propval(pmproperty, value=pmvalue, project=tree[0].name)
#Project.objects.update()
project = Project.objects.get(name=pmproject.name)
property = project.get_property(pmproperty)
#When using the first one this ALWAYS returned None!
self.assertEqual(str(pmvalue), property.str_value)
self.assertEqual(pmvalue, property.value)
Thanks!
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
我在
you 中看到两个问题,
pmPropDef.id
和pmProp.propid
,即使它们相等并且前者不会映射到Property
字段。一般来说,使用
.raw()
查询,您必须返回正确的名称(对于每个字段使用SELECT pmPropDef.name AS name
等) ,或使用可选的转换映射到 raw() 方法,该方法会将列映射到属性。。以下(调整以匹配表中的实际列名称和模型上的字段名称):
如果您确实需要强制值,请尝试在同一查询中强制它。
I see two problems in
you are getting both
pmPropDef.id
andpmProp.propid
, even if they are equal and the former would't map to aProperty
field.in general, using a
.raw()
query, you have to return the correct names (either usingSELECT pmPropDef.name AS name
and so on for each field, or using the optional translation map to the raw() method, which will map columns to properties. It's easy to return directly the actual namesTry the following (adjust to match your actual column names in the table and field names on the model):
If you really need the coerced value already, then try to coerce it in the same query.