VBScript 除法 - 自动将结果四舍五入到小数点后两位

发布于 2024-10-31 01:57:16 字数 290 浏览 0 评论 0原文

我正在尝试将十进制数转换为十六进制数，内置的十六进制函数仅限于 8 个字符，因此我必须编写自己的函数，但是 VBScript 似乎将数字四舍五入到小数点后两位。

例如。

106681252129194 / 16 = 6667578258074.625

问题是当我在 VBScript 中执行此代码时，

strResult = 106681252129194 / 16

结果是 6667578258074.63，这使得计算十六进制数字成为一个小问题。

原文

I'm trying to convert a decimal number to hex, the inbuilt Hex function is limited to 8 characters so I'm having to write my own function, however VBScript seems to be rounding the number to 2 decimal places.

So for example.

106681252129194 / 16 = 6667578258074.625

the problem is when I do it in VBScript this code

strResult = 106681252129194 / 16

the result is 6667578258074.63, which makes calculating the hex digit a slight issue.

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

芸娘子的小脾气 2024-11-07 01:57:16

您必须进行整数除法和余数运算。受邀查看我对 ascii 函数的数字 (double < 2^50) 的尝试：

Function ntoa( nNum, iBase )
  ntoa = "0"
  If nNum Then
     ntoa = Mid( "-", Sgn( nNum ) + 2 ) + ntoaPos( Abs( nNum ), iBase )
  End If
End Function
Function ntoaPos( nNum, iBase )
  If nNum >= 1 Then
     Dim nD : nD = Fix( nNum / iBase )
     Dim nM : nM = nNum - nD * iBase
     ntoaPos =   ntoaPos( nD, iBase ) _
               & Mid( "0123456789ABCDEFGHIJKLMNOPQRSTUV", 1 + nM, 1 )
  End If
End Function

基本测试代码：

  Dim aTests : aTests = Array( _
      "zero (default: long)",           0     _
    , "zero (Byte)"         , CByte(    0   ) _
    , "min_long"            , CLng( -2^31   ) _
    , "max_long"            , CLng(  2^31-1 ) _
    , "min_long - 1"        ,       -2^31-1   _
    , "max_long + 1"        ,        2^31     _
    , "max_ulong"           ,  4294967295     _
    , "big"                 ,        2^40     _
    , "bigger"              ,        2^45     _
    , "bigger-1"            ,        2^45-1   _
    , "bigger-11"           ,        2^45-11   _
    , "very big"            ,        2^50     _
                             )

  Dim nTest
  For nTest = 0 To UBound( aTests ) Step 2
      WScript.Echo "-----", nTest \ 2, aTests( nTest )
      Dim nVal : nVal = aTests( nTest + 1 )
      WScript.Echo "       Type:", TypeName( nVal ), VarType( nVal )
      WScript.Echo "      Value:", nVal
      WScript.Echo "     hexxed:", ntoa( nVal, 16 )
      WScript.Echo "     decced:", ntoa( nVal, 10 )
      WScript.Echo "      abced:", ntoa( nVal, 32 )
  Next

输出：

----- 0 zero (default: long)
       Type: Integer 2
      Value: 0
     hexxed: 0
     decced: 0
      abced: 0
----- 1 zero (Byte)
       Type: Byte 17
      Value: 0
     hexxed: 0
     decced: 0
      abced: 0
----- 2 min_long
       Type: Long 3
      Value: -2147483648
     hexxed: -80000000
     decced: -2147483648
      abced: -2000000
----- 3 max_long
       Type: Long 3
      Value: 2147483647
     hexxed: 7FFFFFFF
     decced: 2147483647
      abced: 1VVVVVV
----- 4 min_long - 1
       Type: Double 5
      Value: -2147483649
     hexxed: -80000001
     decced: -2147483649
      abced: -2000001
----- 5 max_long + 1
       Type: Double 5
      Value: 2147483648
     hexxed: 80000000
     decced: 2147483648
      abced: 2000000
----- 6 max_ulong
       Type: Double 5
      Value: 4294967295
     hexxed: FFFFFFFF
     decced: 4294967295
      abced: 3VVVVVV
----- 7 big
       Type: Double 5
      Value: 1099511627776
     hexxed: 10000000000
     decced: 1099511627776
      abced: 100000000
----- 8 bigger
       Type: Double 5
      Value: 35184372088832
     hexxed: 200000000000
     decced: 35184372088832
      abced: 1000000000
----- 9 bigger-1
       Type: Double 5
      Value: 35184372088831
     hexxed: 1FFFFFFFFFFF
     decced: 35184372088831
      abced: VVVVVVVVV
----- 10 bigger-11
       Type: Double 5
      Value: 35184372088821
     hexxed: 1FFFFFFFFFF5
     decced: 35184372088821
      abced: VVVVVVVVL
----- 11 very big
       Type: Double 5
      Value: 1,12589990684262E+15
     hexxed: 4000000000000
     decced: 1125899906842624
      abced: 10000000000

顺便说一句 - 你不能使用 \:

>> a = 2 ^ 32
>> WScript.Echo a
>> b = a / 2
>> WScript.Echo b
>> c = a \ 2
>> WScript.Echo c
>>
4294967296
2147483648
Error Number:       6
Error Description:  Overflow

You have to do an integer divison and remainder operation. Be invited to check out my attempt at a number (double < 2^50) to ascii function:

Function ntoa( nNum, iBase )
  ntoa = "0"
  If nNum Then
     ntoa = Mid( "-", Sgn( nNum ) + 2 ) + ntoaPos( Abs( nNum ), iBase )
  End If
End Function
Function ntoaPos( nNum, iBase )
  If nNum >= 1 Then
     Dim nD : nD = Fix( nNum / iBase )
     Dim nM : nM = nNum - nD * iBase
     ntoaPos =   ntoaPos( nD, iBase ) _
               & Mid( "0123456789ABCDEFGHIJKLMNOPQRSTUV", 1 + nM, 1 )
  End If
End Function

Basic test code:

  Dim aTests : aTests = Array( _
      "zero (default: long)",           0     _
    , "zero (Byte)"         , CByte(    0   ) _
    , "min_long"            , CLng( -2^31   ) _
    , "max_long"            , CLng(  2^31-1 ) _
    , "min_long - 1"        ,       -2^31-1   _
    , "max_long + 1"        ,        2^31     _
    , "max_ulong"           ,  4294967295     _
    , "big"                 ,        2^40     _
    , "bigger"              ,        2^45     _
    , "bigger-1"            ,        2^45-1   _
    , "bigger-11"           ,        2^45-11   _
    , "very big"            ,        2^50     _
                             )

  Dim nTest
  For nTest = 0 To UBound( aTests ) Step 2
      WScript.Echo "-----", nTest \ 2, aTests( nTest )
      Dim nVal : nVal = aTests( nTest + 1 )
      WScript.Echo "       Type:", TypeName( nVal ), VarType( nVal )
      WScript.Echo "      Value:", nVal
      WScript.Echo "     hexxed:", ntoa( nVal, 16 )
      WScript.Echo "     decced:", ntoa( nVal, 10 )
      WScript.Echo "      abced:", ntoa( nVal, 32 )
  Next

Output:

----- 0 zero (default: long)
       Type: Integer 2
      Value: 0
     hexxed: 0
     decced: 0
      abced: 0
----- 1 zero (Byte)
       Type: Byte 17
      Value: 0
     hexxed: 0
     decced: 0
      abced: 0
----- 2 min_long
       Type: Long 3
      Value: -2147483648
     hexxed: -80000000
     decced: -2147483648
      abced: -2000000
----- 3 max_long
       Type: Long 3
      Value: 2147483647
     hexxed: 7FFFFFFF
     decced: 2147483647
      abced: 1VVVVVV
----- 4 min_long - 1
       Type: Double 5
      Value: -2147483649
     hexxed: -80000001
     decced: -2147483649
      abced: -2000001
----- 5 max_long + 1
       Type: Double 5
      Value: 2147483648
     hexxed: 80000000
     decced: 2147483648
      abced: 2000000
----- 6 max_ulong
       Type: Double 5
      Value: 4294967295
     hexxed: FFFFFFFF
     decced: 4294967295
      abced: 3VVVVVV
----- 7 big
       Type: Double 5
      Value: 1099511627776
     hexxed: 10000000000
     decced: 1099511627776
      abced: 100000000
----- 8 bigger
       Type: Double 5
      Value: 35184372088832
     hexxed: 200000000000
     decced: 35184372088832
      abced: 1000000000
----- 9 bigger-1
       Type: Double 5
      Value: 35184372088831
     hexxed: 1FFFFFFFFFFF
     decced: 35184372088831
      abced: VVVVVVVVV
----- 10 bigger-11
       Type: Double 5
      Value: 35184372088821
     hexxed: 1FFFFFFFFFF5
     decced: 35184372088821
      abced: VVVVVVVVL
----- 11 very big
       Type: Double 5
      Value: 1,12589990684262E+15
     hexxed: 4000000000000
     decced: 1125899906842624
      abced: 10000000000

BTW - you can't use \:

>> a = 2 ^ 32
>> WScript.Echo a
>> b = a / 2
>> WScript.Echo b
>> c = a \ 2
>> WScript.Echo c
>>
4294967296
2147483648
Error Number:       6
Error Description:  Overflow

回复收藏 0 原文

只涨不跌 2024-11-07 01:57:16

您超出了 Double 类型提供的精度，即 8 个字节。

除法实际上并不是自动四舍五入到小数点后两位；而是将结果自动舍入到小数点后两位。只是您的小数点后第三位以上无处可去，因此四舍五入。

一种解决方案是使用Decimal类型，它提供14字节的存储空间。

Dim r As Decimal
r = 106681252129194) / 16 ' r = 6667578258074.625

注：以上内容适用于VB。在 VBA 和 VBScript 中，Decimal 类型只能在 Variant 中访问。您不能直接声明 Decimal 类型的变量；您必须创建一个 Variant，然后使用 CDec 函数将其子类型设置为 Decimal：

Dim r
r = CDec(106681252129194) / CDec(16) ' r = 6667578258074.625

谨慎行事还是有好处的... Decimal 将为您提供 28-29 个有效数字，而 Double 将为您提供 14-15 个有效数字。另一方面，Double 为您提供了在 -324 到 +308 之间的 10 次幂的灵活性，如科学记数法中的那样，而使用 Decimal 时，您只能使用直数符号，没有 10 的幂。

You're exceeding the precision offered by type Double, i.e. 8 bytes.

It isn't really that division auto-rounds things to 2 decimal places; it's just that your 3rd+ decimal places have nowhere to go and are therefore rounded out.

One solution is to use type Decimal, which offers 14 bytes of storage space.

Dim r As Decimal
r = 106681252129194) / 16 ' r = 6667578258074.625

Note: The above is for VB. In VBA and VBScript, the Decimal type is only accessible within a Variant. You can't directly declare a variable of type Decimal; you have to create a Variant and then make its subtype Decimal using the CDec function:

Dim r
r = CDec(106681252129194) / CDec(16) ' r = 6667578258074.625

It's still good to be cautious... Decimal will give you 28-29 significant figures, while Double gives you only 14-15. On the other hand, Double gives you the flexibility of a power of 10 between -324 and +308 as in the scientific notation, while with Decimal you're stuck with straight notation, no powers of 10.

回复收藏 0 原文

~没有更多了~