创建网络错误以测试分布式系统

发布于 2024-12-10 22:14:17 字数 574 浏览 0 评论 0原文

我正在开发一个用于通过 HTTP 进行通信的 Java 库,我想测试其在出现丢包、高延迟、低带宽和拥塞等网络问题时的可靠性和性能。我使用 Apache 的 httpclient 库 从客户端进行连接,以及 Java 自己的com.sun.net.httpserver.HttpServer 用于启动 HTTP 服务器。

是否有库可以做这种事情,或者我应该自己推出?我想我可以尝试将我自己的 org.apache.http.conn.scheme.SchemeSocketFactory 插入客户端,并模拟上面提到的几个问题,但我更喜欢使用已经可以工作的东西:-)

这类似于问题 为单元创建 TCP 网络错误测试,但我正在寻找涉及 Linux 上的 Java 的解决方案。我查看了针对该问题建议的点击,但我不确定它是否可以提供我正在寻找的内容。

I am developing a Java library for communication via HTTP, and I want to test its reliability and performance in case of network problems such as packet loss, high latency, low bandwidth, and congestion. I'm using Apache's httpclient library for doing connections from client side, and Java's own com.sun.net.httpserver.HttpServer for starting up HTTP servers.

Are there libraries available that do that kind of thing, or should I roll my own? I guess I could try to plug my own org.apache.http.conn.scheme.SchemeSocketFactory into the client side, and simulate several of the problems mentioned above with that, but I'd prefer to use something that works already :-)

This is similar to question Creating TCP network errors for unit testing, but I'm looking for solutions involving Java on Linux. I've looked at click, which was suggested for that question, but I'm not sure it can provide what I'm looking for.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

苍白女子 2024-12-17 22:14:17

由于 Java API 无法访问低级网络 API,因此对其进行模拟将具有挑战性。

或者,Apache HTTP Core 组件库可能可以帮助您模拟延迟和数据丢失以及可能的带宽问题。该库具有注入您自己的会话输入和输出缓冲区的功能。您可以在 HTTP Core 的高级功能

通过注入自己的缓冲区

  • 延迟可以是注入到读取和写入中的小延迟
  • 数据丢失是一些字节被丢弃。这不是真正意义上的丢包,因为没有重复发送。
  • 可以通过调整读/写和刷新操作来限制带宽。
  • 除了读取和写入缓冲区的整体减慢之外,流量很难模拟。

您还可以查看 Grinder 项目中提供的 TCPProxy。我没有仔细看过。该文档显示 EchoFilter 作为示例,但我相信应该可以扩展过滤器延迟流中字节的发送和接收。但使用代理来模拟网络问题是有意义的。这样,您的客户端和服务器就可以对正在进行的测试一无所知。

在这两种情况下,网络问题的模拟似乎都是 Java 中可能做到的最大努力。
或者,您可以设置本地防火墙并将其配置为拦截和处理它收到的数据包。

希望这对您有所帮助。

Since Java API's do not have access to the low level network API's, emulating it would be challenging.

Alternatively The Apache HTTP Core components library may have something to help you simulate latency and data loss and possibly bandwidth issues. The library has the feature to inject your own session input and output buffers. You can find the documentation at Advanced Features of HTTP Core

By injecting your own buffer

  • Latency can be a small delay injected into the read and writes
  • Data loss is some bytes thrown away. It is not packet loss in the true sense, since there is no resending of duplicates.
  • Bandwidth can be limited by tuning the read/write and flush operations.
  • Traffic is hard to simulate except as an over all slowdown of reading and writing to the buffer.

You can also look at TCPProxy available in the Grinder project. I haven't looked at it in detail. The documentation shows EchoFilter as an example but I believe it should be possible to extend the filters to delay the sending and receiving of bytes within the stream. But it makes sense to use a proxy to simulate network issues. This way your client and server remain blissfully ignorant of the testing going on.

In both cases simulation of the network issues seems to be the best effort possible in Java.
Or else you can setup a local firewall and have it configured to intercept and play with the packets it receives.

Hope this was helpfull.

薯片软お妹 2024-12-17 22:14:17

如果您不是对系统进行集成测试,IE 与外部服务器一起运行完整的堆栈等等,那么您正在寻找使用模拟工具。它们允许您记录您正在使用的库的行为。这可以节省您运行不需要验证的库代码。

使用 mockito 之类的内容,或者根据需要PowerMock,您可以告诉库在您调用的方法被调用时抛出异常。

示例:

import static org.mockito.Mockito.mock
import static org.mockito.Mockito.when
...
@Test(expected=HttpException.class)
public void invockationThrowsHttpException() {
    ...
    HttpClient httpClient = mock(HttpClient.class)
    when(httpClient).executeMethod(args...).thenThrow(HttpException...)

    underTest.invokeCodeUnderTest(args...)
    ...
}

上面的示例假设 JUnit4。 Mockito 网站上有一个非常好的教程。如果您需要模拟mockito不模拟的东西(例如静态或最终方法),请使用PowerMock。

顺便说一句,我注意到 Apache 的 HTTP 客户端已终止生命周期,他们建议改用 Apache HTTP 组件。如果您的项目仍处于早期阶段,那么现在可能值得切换。

If you are not integration testing the system, I.E running the full stack with external servers and so on, then using mocking tools is what you are looking for. They allow you to record the behaviour of the library you are working against. This save you running the library code, which should not need verification.

Using something like mockito, or if needed PowerMock, you can tell the library to throw exceptions when a method you invoke is invoked.

Example:

import static org.mockito.Mockito.mock
import static org.mockito.Mockito.when
...
@Test(expected=HttpException.class)
public void invockationThrowsHttpException() {
    ...
    HttpClient httpClient = mock(HttpClient.class)
    when(httpClient).executeMethod(args...).thenThrow(HttpException...)

    underTest.invokeCodeUnderTest(args...)
    ...
}

The above example assumes JUnit4. There is a pretty good tutorial on the Mockito website. PowerMock is used if you need to mock things that mockito does not mock (for example static or final methods).

As an aside I noticed that Apache's HTTP Client is on End of Life and that they recommend switching to Apache HTTP Components instead. If your project is still in early stages it may well be worth switching now.

把人绕傻吧 2024-12-17 22:14:17

您应该考虑使用 WAN 模拟器,而不是尝试实现网络错误(在您的情况下,这可能意味着修改 httpclient 库或模拟服务器中的错误和延迟),例如 WANEM。在我看来,这将是一个更简单、更可行的解决方案。

Instead of trying to implement network errors (in your case it could mean to modify the httpclient library or simulating errors and delays in the Server), you should consider to use a WAN Emulator like WANEM. It would be a simpler and more feasible solution, in my opinion.

流心雨 2024-12-17 22:14:17

帖子很老了,但万一有人以某种方式来到这里(就像我所做的那样:)

在集成测试中故意发明网络滞后的方法称为 混沌测试 并因 Netflix 而广为人知,该测试部门开发了一个工具/框架混沌猴子

有一个很酷的项目,使用 Shopify 的 Toxiproxy 方法,它也有 Java 实现 toxiproxy-java

尤其是它与 测试容器,可以模拟微服务网络的一部分甚至整个网络 (⁰o⁰) --> 使用 Toxiproxy 和 Testcontainers 开发弹性应用程序 来自 AtomicJar

希望这能有所帮助并过上幸福的生活,你应得的:)

Post is very old, but in case someone comes here somehow (as I did :)

Methodology of inventing network lags intentionally in integration testing is called Chaos Testing and is widely known thanks to Netflix, which test department has developed a tool/framework Chaos Monkey.

There is a cool project, using this methodology Toxiproxy from Shopify which has Java implementation too toxiproxy-java

Especially it shines with Testcontainers with which part of your microservice network or even a complete one (⁰o⁰) could be simulated --> Developing Resilient Applications with Toxiproxy and Testcontainers from AtomicJar

Hope this helps and live a happy life, you deserve it :)

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文