我有一个 C# 商业应用程序,带有单元测试。使用 NModel 或 Spec Explorer 可以提高可靠性并减少测试时间和费用吗?或者,如果我用 F#(甚至 Haskell)重写它,我可能会看到什么样的(如果有的话)可靠性提高?
代码合同?阿斯麦?
我意识到这是主观的,并且可能是有争议的,所以如果可能的话,请用数据支持你的答案。 :) 或者也许是一个可行的例子,例如埃里克·埃文斯货运系统?
如果我们考虑
单元测试要具体且强大的定理,已检查
准静态地针对特定的“有趣的实例”和类型为一般但弱的定理(通常静态检查),并契约为一般且强的定理,动态检查在常规程序操作期间发生的特定实例。
(来自 B. Pierce 的被认为有害的类型),
这些在哪里其他工具合适吗?
我们可以使用 Java PathFinder、Scala 等对 Java 提出类似的问题。
I've got a business app in C#, with unit tests. Can I increase the reliability and cut down on my testing time and expense by using NModel or Spec Explorer? Alternately, if I were to rewrite it in F# (or even Haskell), what kinds (if any) of reliability increase might I see?
Code Contracts? ASML?
I realize this is subjective, and possibly argumentative, so please back up your answers with data, if possible. :) Or maybe an worked example, such as Eric Evans Cargo Shipping System?
If we consider
Unit tests to be specific and strong theorems, checked
quasi-statically on particular “interesting instances” and Types to be general but weak theorems (usually checked statically), and contracts to be general and strong theorems, checked dynamically for particular instances that occur during regular program operation.
(from B. Pierce's Types Considered Harmful),
where do these other tools fit?
We could pose the analogous question for Java, using Java PathFinder, Scala, etc.
发布评论
评论(2)
可靠性是多个变量的函数,包括软件的总体架构、程序员的能力、需求的质量以及配置管理和一般 QA 流程的成熟度。所有这些都会影响重写的可靠性。
话虽如此,语言确实具有重大影响。在所有其他条件相同的情况下:
将所有这些东西放在一起,您应该拥有一个强大的工具包,可以在整个开发生命周期中提高质量。不幸的是,我不知道有任何强有力的研究能够真正证明这一点。我在开始时列出的所有因素都会混淆任何真正的研究,并且在明确的模式显现出来之前您需要大量数据。
Reliability is a function of several variables, including the general architecture of the software, the capability of the programmers, the quality of the requirements and the maturity of your configuration management and general QA processes. All these will affect the reliability of a rewrite.
Having said that, language certainly has a significant impact. All other things being equal:
Put all of these things together and you should have a powerful toolkit for driving quality through the development lifecycle. Unfortunately I'm not aware of any robust studies that actually prove this. All the factors I listed at the start would confound any real study, and you would need a lot of data before an unambiguous pattern showed itself.
在我的“第一”语言 C# 的上下文中,对引用的一些评论:
是的,但它们可能不会给你一阶逻辑检查,比如“对于所有 x 都存在 ay,其中 f(y)”,更像是“存在 ay,这里是 (!), f(y)”,又名设置、行动、断言。 ;)*
类型不一定那么弱**。
单元测试
Pex + Moles 我认为越来越接近一阶逻辑类型的检查,因为它生成边缘情况并使用 C9 求解器来处理整数约束求解。我真的很想看到更多 Moles 教程(moles 用于替换实现),特别是与某种控制反转容器一起使用,该容器可以利用已经存在的抽象类和接口的存根和实际实现。
弱类型
在C#中,它们相当弱,当然:泛型类型/类型允许您为一个操作添加协议语义——即将类型约束在接口上,这是从某种意义上说,实现类同意的协议。然而,协议的静态类型仅适用于一次操作。
示例:反应式扩展 API
让我们以反应式扩展作为讨论主题。
消费者要求的合约,由可观察者执行。
该协议还有比该接口显示的更多内容:在 IObserver 上调用的方法<在T>实例必须遵循以下协议:
Ordering:
OnNext{0,n} (OnCompleted | OnError){0, 1}
此外,在另一个轴上; time-dimension:
时间:
即,在 OnCompleted x 或 OnError 一次之后,不能调用 OnNext。
此外,并行性轴:
并行性:
即 IObservable 的实现者需要遵循调度约束。没有 IObservable 可以同时从多个线程推送,而不首先在上下文中同步调用。
您如何以简单的方式测试该合约的持有情况?对于c#,我不知道。
API 的消费者
从应用程序的消费端来看,不同上下文之间可能存在交互,例如调度程序、后台/其他线程,并且最好我们希望保证不会结束陷入僵局。
此外,还需要处理可观察量的确定性处理。扩展方法返回的 IObservable 实例何时处理该方法的参数的 IObservable 实例并处理这些实例可能并不总是很清楚,因此需要了解黑匣子的内部工作原理(或者您可以让引用消失)以“合理的方式”,GC 会在某个时候采取它们)
<<<如果没有 Reactive Extensions,这并不一定更容易:
在 TPL 之上实现了任务池。在任务池中,我们有一个工作窃取委托队列,可以在工作线程上调用。
如果我们改变状态,使用 APM/begin/end 或异步模式(排队到任务池)可能会让我们面临回调排序错误。此外,开始调用及其回调的协议可能过于复杂,因此无法遵循。前几天我读到了一篇关于 silverlight 项目的事后分析,该项目在查看所有回调树的业务逻辑森林时遇到问题。然后就有可能实现穷人的异步 monad,即 IEnumerable,其中异步“管理器”迭代它并在每次生成的 IAsyncResult 完成时调用 MoveNext()。
...并且不要让我开始了解 IAsyncResult 中无数的隐藏协议。
不使用响应式扩展的另一个问题是海龟问题 - 一旦您决定希望 IO 阻塞操作是异步的,就需要有海龟一直到 p/invoke 调用将关联的 Win32 线程放置在 IO 完成端口上!如果您有三层,并且最顶层也有一些逻辑,则需要使所有三层都实现 APM 模式;并履行 IAsyncResult 的众多合同义务(或使其部分损坏)——并且基类库中没有默认的公共 AsyncResult 实现。
>>>>>
使用接口中的异常
即使涵盖了上述内存管理+并行性+契约+协议项,仍然有异常需要以良好、可靠的方式处理(不仅仅是接收和忘记)应用。我想举个例子;
上下文
假设我们发现自己从契约/接口捕获了一个异常(不一定是从反应式扩展的 IObservable 实现中捕获的,它具有单子异常处理而不是基于堆栈帧)。
希望程序员很勤奋并记录了可能的异常,但自始至终都可能存在异常的可能性。如果一切都用代码契约正确定义,至少我们可以确定我们能够捕获一些异常,但是许多不同的原因可能会集中在一个异常类型中,一旦抛出异常,我们如何确保纠正尽可能少的工作?
目标
假设我们正在从应用程序中的消息总线消费者推送一些数据记录,并在后台线程上接收它们,该后台线程决定如何处理它们。
示例
一个现实生活中的例子是 Spotify,我每天都在使用它。
我的 100 美元路由器/接入点有时会认输。我猜它有一个缓存错误或某种堆栈溢出错误,因为每次我通过它推送超过 2 MB/s 的 LAN/WAN 数据时都会发生这种情况。
我必须把网卡注册起来;无线网络和以太网卡。以太网连接断开。 Spotify 事件处理程序循环的套接字返回无效代码(我认为是 C 或 C++)或抛出异常。 Spotify 必须处理它,但它不知道我的网络拓扑是什么样的(并且没有代码来尝试所有路由/更新路由表以及要使用的接口);我仍然有通往互联网的路线,但只是不在同一个界面上。 Spotify 崩溃了。
论文
例外在语义上根本不够。我相信可以从Haskell中Error monad的角度来看待异常。我们要么继续,要么中断:展开堆栈、执行 catch、执行finally,祈祷我们不会在其他异常处理程序或 GC 上出现竞争条件,或者在未完成的 IO 完成端口上出现异步异常。
但是,当我的某个接口的连接/路由出现故障时,Spotify 就会崩溃并冻结。
现在我们有了 SEH/结构化异常处理,但我认为将来我们将会有 SEH2,其中每个异常源与实际异常一起给出一个可区分的联合(即它应该静态类型化到链接库/程序集),可能的补偿操作 - 在这个示例中,我可以想象 Windows 的网络 API 告诉应用程序执行补偿操作以在另一个接口上打开相同的套接字,或者自行处理它(就像现在一样),或者重试套接字,具有一些内核管理的重试策略。这些选项中的每一个都是可区分联合类型的一部分,因此实现者必须使用其中之一。
我认为,当我们有了 SEH2 时,就不会再被称为例外了。
^^
无论如何,我已经离题太多了。
与其阅读我的想法,不如听听 Erik Meijer 的一些想法 - 这是一个非常好的 他和 Joe Duffy 之间的圆桌讨论。他们讨论了处理调用的副作用。或者查看此搜索列表。
今天,我发现自己处于一个位置,作为一名顾问,维护一个系统,其中更强的静态语义可能会更好,并且我正在寻找可以为我提供编程速度+正确性验证的工具,其级别为准确且精确。我还没找到。
我只是认为,距离面向开发人员的可靠计算,我们还需要 20 年(甚至更多)。现在有太多的语言、框架、营销废话和概念,普通开发人员无法掌握这些知识。
为什么这是在“弱类型”的标题下?
因为我发现类型系统将成为解决方案的一部分;类型不必弱!简洁的代码和强类型系统(比如 Haskell)可以帮助程序员构建可靠的软件。
Some comments on the quote, in the context of C# which is my "first" language:
Yes, but they might not give you first order logic checks, like "for all x there exists a y where f(y)", more like "there exists a y, here it is (!), f(y)", aka setup, act, assert. ;)*
Types are not necessarily that weak**.
Unit Testing
Pex + Moles I think is getting closer to the first-order logic type of checking, as it generates the edge-cases and uses the C9 solver to work with integer constraint solving. I would really like to see more Moles tutorials (moles is for replacing implementations), specifically together with some sort of inversion of control container that can leverage what stub- and real- implementations of abstract classes and interfaces already exist.
Weak Types
In C# they are fairly weak, sure: generic typing/types allows you to add protocol semantics for one operation -- i.e. constraining types to be on interfaces, which are in some sense protocols which implementing classes agree to. However, the static typing of the protocol is just for one operation.
Example: Reactive Extensions API
Let's take Reactive Extensions as a discussion topic.
The contract required by the consumer, implemented by the observable.
There are more to the protocol than this interface shows: methods called on an IObserver< in T > instance must follow this protocol:
Ordering:
OnNext{0,n} (OnCompleted | OnError){0, 1}
Furthermore, on another axis; time-dimension:
Time:
i.e. no invocation to OnNext may be done after one to OnCompleted xor OnError.
Furthermore, the axis of parallelism:
Parallelism:
i.e. there's a scheduling constraint that needs to be followed from implementers of IObservable. No IObservable may push from multiple threads at the same time, without first synchronizing the invocation around a context.
How do you test this contract holds in an easy way? With c#, I don't know.
Consumer of API
From the consuming side of the application, there might be interactions between different contexts, such as Dispatcher, Background/other threads, and preferably we'd like to give guarantees that we don't end up in a deadlock.
Further, there is the requirement to handle deterministic disposing of the observables. It might not be clear all the time when an extension method's returned IObservable instance takes care of the method's arguments' IObservable instances and dispose those, so there's a requirement to know about the inner workings of the black box (alternatively you can let the references go in a "reasonable way" and the GC will take them at some point)
<<< Without Reactive Extensions, it's not necessarily easier:
There is the task pool on top of TPL is implemented. In the task pool we have a work-stealing queue of delegates to invoke on the worker threads.
Using the APM/begin/end or the async pattern (which queues to the task pool) could leave us open to callback-ordering bugs if we mutating state. Also, the protocol of begin-invocations and their callbacks might be too convoluted and hence impossible to follow. I read a post-mortem the other day about a silverlight project having problems seeing the business logic-forest for all the callback-trees. Then there's the possibility of implementing the poor-man's async monad, the IEnumerable with an async 'manager' iterating through it and calling MoveNext() every time a yielded IAsyncResult completes.
...and don't get me started on the nuuuumerous hidden protocols in IAsyncResult.
Another problem, without using Reactive extensions is the turtles problem - once you decide that you want an IO-blocking operation to be async, there need to be turtles all the way down to the p/invoke call that places the associated Win32-thread on an IO-completion port! If you have three layers and then some logic as well inside of your topmost layer, you need to make all three layers implement the APM pattern; and fulfil the numerous contract obligations of IAsyncResult (or leave it partially broken) -- and there's no default public AsyncResult implementation in the base class library.
>>>
Working with exceptions from the interface
Even with the above memory-management + parallelism + contract + protocol items covered, there are still exceptions to be handled (not just received and forgotten about), in a good, reliable application. I want to make an example;
Context
Let's say that we find ourselves catching an exception from the contract/interface (not necessarily from reactive extensions' IObservable implementations here which have monadic exception handling rather than stack-frame based).
Hopefully the programmer was diligent and documented the possible exceptions, but there might be exception possibilities all the way down. If everything is correctly defined with code contracts at least we can be sure we are capable of catching a few of the exceptions, but many different causes may be lumped together inside of one exception type, and once an exception is thrown, how do we ensure that the work of the least possible size is rectified?
Aim
Say that we are pushing some data-record from a message-bus-consumer in our application, and receiving them on the background thread which decides what to do with them.
Example
A real-life example here could be Spotify, which I'm using every day.
My $100 router/access point throws in the towel at random times. I guess it has a cache-bug or some sort of stack overflow bug, as it happens every time I push more than 2 MB/s LAN/WAN data through it.
I have to NICs up; the wifi and the ethernet card. Ethernet's connection goes down. The sockets of Spotify's event-handler loop return an invalid code (I think it's C or C++) or throw exceptions. Spotify has to handle it, but it doesn't know what my network topology looks like (and there is no code to try all routes/update the routing table and hence the interface to be used); I still have a route to the internet, but just not on the same interface. Spotify crashes.
A thesis
Exceptions are simply not semantic enough. I believe one can look at exceptions from the perspective of the Error monad in Haskell. We either continue or break: unwinding the stack, executing the catches, executing the finally's an praying we don't end up with race conditions on either other exception handlers or the GC, or async exceptions for outstanding IO-completion ports.
But when one of my interfaces' connection/route goes down, Spotify crashes freezes.
Now we have SEH/Structured Exception Handling, but I think we will have SEH2 in the future, where each source of exception gives, with the actual exception, a discriminated union (i.e. it should be statically typed to the linked library/assembly), of possible compensating actions -- in this example, I could imagine Windows' network API telling the application to execute a compensating action to open the same socket on another interface, or to handle it on its own (like now), or to retry the socket, with some kernel-managed retry policy. Each of these options are parts of a discriminated union type, so the implementer must use one of them.
I think that, when we have SEH2, it won't be called exceptions anymore.
^^
Anyway, I have digressed too much already.
Instead of reading my thoughts, listen to some of Erik Meijer's -- this is a very good round-table discussion between him and Joe Duffy. They discuss handling side-effects of calls. Or have a look at this search listing.
I'm finding myself in a position, today, as a consultant, of maintaining a system where stronger static semantics could be good, and I'm looking at tools which can give me the speed of programming + the correctness verification on a level which is accurate and precise. I haven't found it yet.
I simply think we are another 20 years if not more away from developer oriented reliable computing. There are just too many languages, frameworks, marketing BS and concepts in the air right now, for the ordinary develop to stay on top of things.
Why is this under the heading of "weak types"?
Because I find that the type system will be part of the solution; types need not be weak! Terse code and strong type systems (think Haskell) help programmers build reliable software.