进行大量查询时，asp.net 网站停止响应
我有一个即将发布的 asp.net 网站。该网站涉及企业管理。我为发布做的准备工作之一是将购买的企业列表中的一大堆数据上传到我们的 SQL 数据库中。为此，我使用了一种非常简单的方法：将 .csv 文件放在服务器上的文件夹中，并使用临时页面上的 Page_Load 事件来执行处理数据所需的逻辑并将其放入数据库中。
这是一个漫长的过程。业务列表被分解为一大堆文件，我在页面上设置了脚本，以便在每次刷新页面时从一个文件导入所有数据。每个文件大约需要 10 分钟。
奇怪的是：当处理这些数据时（当服务器正在处理所有内容时，在 page_load 方法终止之前）网站被阻止 - 没有其他用户可以访问它，并且网站上的其他页面也不会加载。他们陷入了“正在连接...”阶段。
目前这不是一个大问题，因为该网站只是在测试环境中运行。但是，当站点上线时，我可能需要使用类似的脚本将数据放入数据库，显然在发生这种情况时阻止所有用户的访问是不可接受的。
我相当确定这不是 SQL Server 的问题，因为我能够在该进程运行时同时进行查询，因此肯定是 Web 服务器出了问题。
我无法发布代码摘录，因为它冗长且技术性强。不过，这并不困难：该方法从 .csv 文件中读取数据，对其进行处理，并对文件的每一行向外部服务发出 HTTP 请求，以收集一些更相关的数据。然后，所有数据被插入到SQL数据库中。
据我了解，网络服务器不应该像这样阻塞。他们应该能够同时向大量用户提供页面，即使一个线程/用户需要大量资源。
任何人都可以建议可能会发生什么吗？我应该从哪里开始寻找解决方案？
I have an asp.net website that is nearing release. The site deals with business management. One of the things I am doing to prepare for release is upload a whole bunch of data from purchased lists of businesses into our SQL database. To do this, I'm using a very simple method: I put the .csv files in a folder on the server, and use the Page_Load event on a temporary page to do the logic needed to process the data and put it in our database.
This is a lengthy process. the business lists are broken up into a whole bunch of files, and I've got my script on the page set up to import all the data from one file on each refresh of the page. It takes about 10 minutes per file.
The weird thing is this: When processing this data ( while the server is processing everything, before the page_load method terminates) the website is blocked - no other users can access it, and no other pages on the site will load. They get stuck on the "Connecting..." phase.
This isn't a huge problem right now, as the site is just running in a test environment. However, I may need to use similar scripts to put data into the database when the site is live, and obviously its not acceptable to have access to all users blocked when this is happening.
I am fairly sure this isn't a problem with the SQL Server, as I am able to do simultaneous queries while this process is running, so it must be something wrong with the web server.
I can't post a code exerpt, because it is long winded and technical. However, nothing difficult is going on: The method reads in the data from the .csv file, processes it, and makes an HTTP Request to an external service for each line of the file, to collect some more relevant data. Then, all the data is inserted into the SQL database.
As I understand it, web servers shouldn't block like this. They should be able to simulatenously serve pages to lots of users at once, even if one thread/user is requiring lots of resources.
Can any suggest what might be up? Where should I start looking for a solution?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
让异步处理程序处理请求。
http://msdn.microsoft.com/en -us/library/ms227433%28v=vs.85%29.aspx
这将允许您的网站在长时间进程运行时继续开展业务。
Have the request be processed by an asynchrounous handler.
http://msdn.microsoft.com/en-us/library/ms227433%28v=vs.85%29.aspx
This will allow your website to continue doing it's business while the long process is running.
ASP.NET 不适合繁重的处理。
ASP.NET 页面应该只触发该操作。后台进程应该承担繁重的工作。两者都可以使用队列或类似的东西进行通信。
网页应该简单地报告后台任务的状态。
这种范式是现代计算（尤其是云计算）的核心。
ASP.NET is not meant for heavy processing.
The ASP.NET page should just trigger the action. A background process should do the heavy lifting. Both can communicate using a queue or something similar.
The Web page should simply report the status of the background tasks.
This paradigm is central in modern computing (especially cloud computing).
请参阅http://msdn.microsoft.com/en-us/来自msdn的library/ff647787.aspx#scalenetchapt06_topic8。
我认为问题在于您使用的是 asp.net 的默认设置，您没有获得足够的线程并且只有 2 个出站 tcp/ip 连接。这对于一个真正的网站来说太可怕了。
请更改 machine.config 中的 maxconnections、maxworkerthreads 和其他内容。其中一些设置有一个公式，例如 12 * CPU 数量等。
这应该会给您带来所需的性能提升。
希望这有帮助。
编辑：-
我的评论是在我了解 machine.config 中的 autoconfig=true 设置之前发表的。我做了一些初步研究，发现这些设置适用于大多数应用程序。因此，这些自定义设置仅适用于可能需要仔细考虑优化的特殊场景。
Please refer to http://msdn.microsoft.com/en-us/library/ff647787.aspx#scalenetchapt06_topic8 from msdn.
I think the problem is that you are using the default settings for asp.net where you don't get enough threads and only 2 outbound tcp/ip connections. This is just horrible for a real website.
Please change maxconnections, maxworkerthreads and others in your machine.config. There is a formula for some of these settings like 12 * number of CPUs, etc.
This should give you the performance boost you need.
Hope this helps.
Edit:-
My comments were before I learnt about the autoconfig=true setting in the machine.config. I did some preliminary research and discovered that these settings work for most applications. so, these customized settings are only needed for specialized scenarios that might need carefully considered optimizations.