Transaction / RawSequel - 以及测试的乐趣

发布于 2024-11-29 02:42:54 字数 4830 浏览 0 评论 0原文

这两者都适用于“生产”,后者仅适用于测试。现在我已经在生产和测试中得到了一些东西,我想了解为什么我必须采用整个游标路线而不是 Django 路线。我相信问题与交易有关,但我并不乐观,因为我晚上 8:30 坐在这里,这让我很烦恼。

这与此问题相关我以为我已经有了答案(和理解),但可惜我没有。我的测试是 A/B,其中 A 在 Django 外部注入,B 与已知的 A 进行比较。提供的答案解决了我的部分问题,但当我添加更多测试时,问题不断重新出现。

我深入研究并假设 RawQuery 没有提交事务,但没有金额 transaction.commit 似乎解决了这个问题。我还从中删除了 django.testing.TestCase 并直接进行单元测试。我想我已经尝试了每种组合,但我对 SQL 或事务支持不是很精通,现在我想知道为什么一种有效而另一种无效...

如果有人在这里有任何见解,我真的会感谢!

更新 2 已修改并清理,但仍然失败..

    # BUG: https://code.djangoproject.com/ticket/12768
    # - Requirement for pmProp.* - This (in-part) forced me to shift to raw.
    sqlraw = """ SELECT
                    pmProp.propid as propid_id,
                    pmProp.owner as owner,
                    pmProp.ownertype as ownertype,
                    pmProp.behavior as behavior,
                    pmProp.value as value_id,
                    pmPropDef.id as propdef_id,
                    pmPropDef.name as name,
                    pmPropDef.datatype as datatype,
                    pmPropDef.choicetype as choicetype,
                    pmPropDef.definition as definition_id,
                    pmPropDef.ptrig as prop_trigger,
                    pmPropDef.units as units,
                    IFNULL(pmPropShort.str, pmPropLong.str) as str_value FROM pmProp
                INNER JOIN pmPropDef ON pmProp.propid=pmPropDef.id AND pmPropDef.name=%s
                LEFT JOIN pmPropShort ON sid=pmProp.value
                LEFT JOIN pmPropLong ON lid=-pmProp.value
                WHERE pmProp.ownertype=%s AND pmProp.owner=%s AND pmPropDef.id=pmProp.propid
                """
    if explicit:
            sqlraw += " AND pmProp.behavior='explicit'"

    # TRY ONE - DOES NOT WORK FOR TESTING..
    # This will NOT work for testing - It simply doesn't get the value
    # when repeatedly inserting from pm and checking the value.
    #
    # Note if you use this you must update the sqlraw to include pmProp.* bug..
    #
    #try:
    #    result = list(Property.objects.raw(sqlraw, [property, owner, self.id]))[0]
    #    result.value = self.coerce_datatype(result.str_val, result.datatype)
    #except IndexError:
    #    result = None
    # END TRY ONE

    # Try TWO:  THIS WORKS for both
    cursor = connections['catalog'].cursor()
    cursor.execute(sqlraw, [property, owner, self.id])
    row = cursor.fetchone()
    transaction.commit_unless_managed(using='catalog')

    if row:
        field_map =  "propid_id owner ownertype behavior value_id propdef_id "
        field_map += "name datatype choicetype definition_id prop_trigger "
        field_map +=  "units str_value"
        field_map = field_map.split()
        class PropVal(object): pass
        result = PropVal()
        result.__dict__=dict(zip(field_map, row))
        result.value = self.coerce_datatype(result.str_value, result.datatype)
        try:
            log.info("%s %s=%s %s" % (property.capitalize(), result.name,
                                               result.value, result.units))
        except UnicodeDecodeError: pass
    else:
        result = None
    # END TRY Two

更新

这是一个示例 A/B 测试。

from django.db import connection, transaction
from unittest import TestCase
#from django.test import TestCase, TransactionTestCase
from apps.pmCatalog.utility.ICMPM.pm import Pm
from apps.pmCatalog.models import Property, Site, Project, Variant, Library, LibraryType, Release
import settings
import datetime

import logging
log = logging.getLogger(__name__)

class PropertyTests(TestCase):

    def test_add_property_value(self):
        """Test the ability to add a property and retrieve a property"""

        prop_types = [("string", "Funny Business"), ("integer", 1234), ("real", 12.34) ]
        pm = Pm(mysql_db='test_bugs')
        tree = pm.add_release_tree()

        for prop_type, pmvalue in prop_types:
            # Add a property definition for a branch (like a project)
            pmproperty = "%s_%s_basic" % (tree[0].name, prop_type)
            pm.add_property_definition(pmproperty, prop_type=prop_type)
            pm.add_propval(pmproperty, value=pmvalue, project=tree[0].name)
            #Project.objects.update()
            project = Project.objects.get(name=pmproject.name)
            property = project.get_property(pmproperty)
            #When using the first one this ALWAYS returned None!
            self.assertEqual(str(pmvalue), property.str_value)
            self.assertEqual(pmvalue, property.value)

谢谢!

Both of these work in "production" The later only works in testing. Now that I've gotten something to work in both production and testing I'd like to understand why I had to go the whole cursors route rather than the Django route. I believe the problem has to do with transactions but I'm not positive and as I sit here at 8:30pm it's bugging me.

This is related to this question where I thought I had my answer (and an understanding) but alas I didn't. My testing is an A/B where A is injected outside of Django and B compares against the known A. The answer that was provided solved part of my problem but when I added in more tests the problem kept resurfacing.

I dug in and assumed it was the RawQuery was not committing the transaction but no amount transaction.commit seemed to fix it. I also removed the django.testing.TestCase from it and went straight unittest. I think I've tried every combination but I'm not very proficient in SQL or Transactional support and now I'm left wondering why one works and one doesn't...

If anyone has any insight here I would really appreciate it!

Update 2 Revised and cleaned up but still failing..

    # BUG: https://code.djangoproject.com/ticket/12768
    # - Requirement for pmProp.* - This (in-part) forced me to shift to raw.
    sqlraw = """ SELECT
                    pmProp.propid as propid_id,
                    pmProp.owner as owner,
                    pmProp.ownertype as ownertype,
                    pmProp.behavior as behavior,
                    pmProp.value as value_id,
                    pmPropDef.id as propdef_id,
                    pmPropDef.name as name,
                    pmPropDef.datatype as datatype,
                    pmPropDef.choicetype as choicetype,
                    pmPropDef.definition as definition_id,
                    pmPropDef.ptrig as prop_trigger,
                    pmPropDef.units as units,
                    IFNULL(pmPropShort.str, pmPropLong.str) as str_value FROM pmProp
                INNER JOIN pmPropDef ON pmProp.propid=pmPropDef.id AND pmPropDef.name=%s
                LEFT JOIN pmPropShort ON sid=pmProp.value
                LEFT JOIN pmPropLong ON lid=-pmProp.value
                WHERE pmProp.ownertype=%s AND pmProp.owner=%s AND pmPropDef.id=pmProp.propid
                """
    if explicit:
            sqlraw += " AND pmProp.behavior='explicit'"

    # TRY ONE - DOES NOT WORK FOR TESTING..
    # This will NOT work for testing - It simply doesn't get the value
    # when repeatedly inserting from pm and checking the value.
    #
    # Note if you use this you must update the sqlraw to include pmProp.* bug..
    #
    #try:
    #    result = list(Property.objects.raw(sqlraw, [property, owner, self.id]))[0]
    #    result.value = self.coerce_datatype(result.str_val, result.datatype)
    #except IndexError:
    #    result = None
    # END TRY ONE

    # Try TWO:  THIS WORKS for both
    cursor = connections['catalog'].cursor()
    cursor.execute(sqlraw, [property, owner, self.id])
    row = cursor.fetchone()
    transaction.commit_unless_managed(using='catalog')

    if row:
        field_map =  "propid_id owner ownertype behavior value_id propdef_id "
        field_map += "name datatype choicetype definition_id prop_trigger "
        field_map +=  "units str_value"
        field_map = field_map.split()
        class PropVal(object): pass
        result = PropVal()
        result.__dict__=dict(zip(field_map, row))
        result.value = self.coerce_datatype(result.str_value, result.datatype)
        try:
            log.info("%s %s=%s %s" % (property.capitalize(), result.name,
                                               result.value, result.units))
        except UnicodeDecodeError: pass
    else:
        result = None
    # END TRY Two

Update

Here is a sample A/B test.

from django.db import connection, transaction
from unittest import TestCase
#from django.test import TestCase, TransactionTestCase
from apps.pmCatalog.utility.ICMPM.pm import Pm
from apps.pmCatalog.models import Property, Site, Project, Variant, Library, LibraryType, Release
import settings
import datetime

import logging
log = logging.getLogger(__name__)

class PropertyTests(TestCase):

    def test_add_property_value(self):
        """Test the ability to add a property and retrieve a property"""

        prop_types = [("string", "Funny Business"), ("integer", 1234), ("real", 12.34) ]
        pm = Pm(mysql_db='test_bugs')
        tree = pm.add_release_tree()

        for prop_type, pmvalue in prop_types:
            # Add a property definition for a branch (like a project)
            pmproperty = "%s_%s_basic" % (tree[0].name, prop_type)
            pm.add_property_definition(pmproperty, prop_type=prop_type)
            pm.add_propval(pmproperty, value=pmvalue, project=tree[0].name)
            #Project.objects.update()
            project = Project.objects.get(name=pmproject.name)
            property = project.get_property(pmproperty)
            #When using the first one this ALWAYS returned None!
            self.assertEqual(str(pmvalue), property.str_value)
            self.assertEqual(pmvalue, property.value)

Thanks!

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

层林尽染 2024-12-06 02:42:54

我在

you 中看到两个问题,

sqlraw = """SELECT pmProp.*, pmPropDef.id, pmPropDef.name, pmPropDef.units,
            IFNULL(pmPropShort.str, pmPropLong.str) as value FROM pmProp
            INNER JOIN pmPropDef ON pmProp.propid=pmPropDef.id AND pmPropDef.name=%s
            LEFT JOIN pmPropShort ON sid=pmProp.value
            LEFT JOIN pmPropLong ON lid=-pmProp.value
            WHERE pmProp.ownertype=%s AND pmProp.owner=%s AND pmPropDef.id = pmProp.propid
            """
  1. 即同时获得 pmPropDef.idpmProp.propid,即使它们相等并且前者不会映射到 Property 字段。

  2. 一般来说,使用 .raw() 查询,您必须返回正确的名称(对于每个字段使用 SELECT pmPropDef.name AS name 等) ,或使用可选的转换映射到 raw() 方法,该方法会将列映射到属性。

。以下(调整以匹配表中的实际列名称和模型上的字段名称):

sqlraw = """SELECT 
                pmProp.id as id,
                pmProp.owner as owner,
                pmProp.ownertype as ownertype,
                pmProp.behavior as behavior,
                pmProp.propdef_id as propdef_id,
                pmPropDef.name as name, 
                pmPropDef.units as units,
                IFNULL(pmPropShort.str, pmPropLong.str) as str_value
            FROM pmProp
            INNER JOIN pmPropDef ON pmProp.propid=pmPropDef.id AND pmPropDef.name=%s
            LEFT JOIN pmPropShort ON sid=pmProp.value
            LEFT JOIN pmPropLong ON lid=-pmProp.value
            WHERE pmProp.ownertype=%s AND pmProp.owner=%s AND pmPropDef.id = pmProp.propid
            """

如果您确实需要强制值,请尝试在同一查询中强制它。

I see two problems in

sqlraw = """SELECT pmProp.*, pmPropDef.id, pmPropDef.name, pmPropDef.units,
            IFNULL(pmPropShort.str, pmPropLong.str) as value FROM pmProp
            INNER JOIN pmPropDef ON pmProp.propid=pmPropDef.id AND pmPropDef.name=%s
            LEFT JOIN pmPropShort ON sid=pmProp.value
            LEFT JOIN pmPropLong ON lid=-pmProp.value
            WHERE pmProp.ownertype=%s AND pmProp.owner=%s AND pmPropDef.id = pmProp.propid
            """
  1. you are getting both pmPropDef.id and pmProp.propid, even if they are equal and the former would't map to a Property field.

  2. in general, using a .raw() query, you have to return the correct names (either using SELECT pmPropDef.name AS name and so on for each field, or using the optional translation map to the raw() method, which will map columns to properties. It's easy to return directly the actual names

Try the following (adjust to match your actual column names in the table and field names on the model):

sqlraw = """SELECT 
                pmProp.id as id,
                pmProp.owner as owner,
                pmProp.ownertype as ownertype,
                pmProp.behavior as behavior,
                pmProp.propdef_id as propdef_id,
                pmPropDef.name as name, 
                pmPropDef.units as units,
                IFNULL(pmPropShort.str, pmPropLong.str) as str_value
            FROM pmProp
            INNER JOIN pmPropDef ON pmProp.propid=pmPropDef.id AND pmPropDef.name=%s
            LEFT JOIN pmPropShort ON sid=pmProp.value
            LEFT JOIN pmPropLong ON lid=-pmProp.value
            WHERE pmProp.ownertype=%s AND pmProp.owner=%s AND pmPropDef.id = pmProp.propid
            """

If you really need the coerced value already, then try to coerce it in the same query.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文