.net-并发HttpWebRequests的最大数量

我正在对一个Web应用程序进行压力测试,并建立了一个Windows测试程序,该程序启动了多个线程,并在每个线程上发送了一个Web请求。

问题是我得到以下输出:

01/09/09 11:34:04 Starting new HTTP request on 10
01/09/09 11:34:04 Starting new HTTP request on 11
01/09/09 11:34:04 Starting new HTTP request on 13
01/09/09 11:34:05 Starting new HTTP request on 14
01/09/09 11:34:05 Starting new HTTP request on 11
01/09/09 11:34:05 11 has finished!
01/09/09 11:34:05 Starting new HTTP request on 13
01/09/09 11:34:05 13 has finished!
01/09/09 11:34:05 Starting new HTTP request on 14
01/09/09 11:34:05 14 has finished!
01/09/09 11:34:05 Starting new HTTP request on 11
01/09/09 11:34:05 11 has finished!
01/09/09 11:34:05 Starting new HTTP request on 14
01/09/09 11:34:05 14 has finished!
01/09/09 11:34:05 Starting new HTTP request on 13
01/09/09 11:34:05 13 has finished!
01/09/09 11:34:05 Starting new HTTP request on 15
01/09/09 11:34:06 Starting new HTTP request on 11
01/09/09 11:34:06 11 has finished!
01/09/09 11:34:06 Starting new HTTP request on 14
01/09/09 11:34:06 14 has finished!

即使我这样创建了100个,最多也有5个线程:

int numberOfThreads = Convert.ToInt32(txtConcurrentThreads.Text);

    List<BackgroundWorker> workers = new List<BackgroundWorker>();

    for (int N = 0; N < numberOfThreads; N++)
    {

        BackgroundWorker worker = new BackgroundWorker();
        worker.DoWork += new DoWorkEventHandler(worker_DoWork);
        worker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(worker_RunWorkerCompleted);
        workers.Add(worker);
    }


    foreach(BackgroundWorker worker in workers)
    {
        worker.RunWorkerAsync();
    }

有人能启发我发生什么事吗?

谢谢

编辑:如果按照建议我睡了5秒钟,而不是httpwebrequest,那么我确实会触发更多的线程,但没有我期望的那么多:

01/09/09 11:56:14 Starting new HTTP request on 7
01/09/09 11:56:14 Starting new HTTP request on 11
01/09/09 11:56:15 Starting new HTTP request on 12
01/09/09 11:56:15 Starting new HTTP request on 13
01/09/09 11:56:16 Starting new HTTP request on 14
01/09/09 11:56:16 Starting new HTTP request on 15
01/09/09 11:56:17 Starting new HTTP request on 16
01/09/09 11:56:17 Starting new HTTP request on 17
01/09/09 11:56:18 Starting new HTTP request on 18
01/09/09 11:56:19 Starting new HTTP request on 7
01/09/09 11:56:19 7 has finished!
01/09/09 11:56:19 Starting new HTTP request on 11
01/09/09 11:56:19 11 has finished!
01/09/09 11:56:19 Starting new HTTP request on 19
01/09/09 11:56:20 Starting new HTTP request on 20
01/09/09 11:56:20 Starting new HTTP request on 12
01/09/09 11:56:20 12 has finished!

看起来我每秒只有2个线程,这对我来说似乎太慢了。 我想Console.WriteLine可能是个问题?

编辑:我设置

ThreadPool.SetMinThreads(100, 4); 

System.Net.ServicePointManager.DefaultConnectionLimit = 100;

并得到以下结果:

01/09/09 14:00:07 Starting new HTTP request on 11
01/09/09 14:00:07 Starting new HTTP request on 81
01/09/09 14:00:07 Starting new HTTP request on 82
01/09/09 14:00:07 Starting new HTTP request on 79
01/09/09 14:00:07 Starting new HTTP request on 83
01/09/09 14:00:07 Starting new HTTP request on 84
01/09/09 14:00:07 Starting new HTTP request on 85
01/09/09 14:00:07 Starting new HTTP request on 87
01/09/09 14:00:07 Starting new HTTP request on 88
...
01/09/09 14:00:07 84 has finished! Took 323.0323 milliseconds
01/09/09 14:00:08 88 has finished! Took 808.0808 milliseconds
01/09/09 14:00:08 96 has finished! Took 806.0806 milliseconds
01/09/09 14:00:08 94 has finished! Took 806.0806 milliseconds
01/09/09 14:00:08 98 has finished! Took 801.0801 milliseconds
01/09/09 14:00:08 80 has finished! Took 799.0799 milliseconds
01/09/09 14:00:08 86 has finished! Took 799.0799 milliseconds
01/09/09 14:00:08 92 has finished! Took 799.0799 milliseconds
01/09/09 14:00:08 100 has finished! Took 812.0812 milliseconds
01/09/09 14:00:08 82 has finished! Took 1010.101 milliseconds

因此能够同时发出大量Web请求。 似乎要排队(调出STA COM +服务器),所以这就是我的期望。

谢谢你的帮助

Duncan asked 2019-10-09T10:37:51Z
4个解决方案
68 votes

您的所有(或大部分)请求是否都偶然发送到同一主机? 每个主机都有一个内置限制。 您可以在system.Net connectionManagement元素的app.config中进行更改。

另一件事是线程池仅逐渐增加其线程数-它每半秒启动一个新线程,即IIRC。 那可能是您所看到的吗? 尝试摆脱方程式中的HttpWebRequest-而是睡几秒钟...

我怀疑后一个问题是您最初遇到的一个问题,但是第一个问题也会引起您的问题。

Jon Skeet answered 2019-10-09T10:38:17Z
51 votes

同时传出HTTP连接的数量受到限制。 我认为您可以在创建HttpWebRequest对象之前通过使用HttpWebRequest静态属性进行控制。

Fredrik Mörk answered 2019-10-09T10:38:42Z
5 votes

我对.NET Core的了解还很少。 .NET Core 1中没有包含ServicePointManager,但在版本2中似乎又回来了。但是,对于HttpClient,您还可以这样设置最大连接数:

new HttpClient(new HttpClientHandler
                {
                    MaxConnectionsPerServer = 100
                })
cbp answered 2019-10-09T10:39:06Z
4 votes

如果我在Windows config中写以下标记,则每次执行以下代码时都会执行该标记。 因此,每次执行下面的代码时,我将被允许最多具有1000000个并行请求/响应数。

HttpWebRequest POSTRequest = (HttpWebRequest)WebRequest.Create("http://yahoo.com");

标签

<connectionManagement>
 <clear/>
 <add address="*" maxconnection="1000000" />
</connectionManagement>
user220550 answered 2019-10-09T10:39:37Z
translate from https://stackoverflow.com:/questions/1361771/max-number-of-concurrent-httpwebrequests