Python中文
首页
教程
问答
标签
搜索
登录
注册
Django/uwsgi性能差
回答此问题可获得
20
贡献值,回答如果被采纳可获得
50
分。
<p>我正在用nginx&uwsgi运行一个django应用程序。以下是我如何运行uwsgi:</p> <pre><code>sudo uwsgi -b 25000 --chdir=/www/python/apps/pyapp --module=wsgi:application --env DJANGO_SETTINGS_MODULE=settings --socket=/tmp/pyapp.socket --cheaper=8 --processes=16 --harakiri=10 --max-requests=5000 --vacuum --master --pidfile=/tmp/pyapp-master.pid --uid=220 --gid=499 </code></pre> <p>nginx配置:</p> <pre><code>server { listen 80; server_name test.com root /www/python/apps/pyapp/; access_log /var/log/nginx/test.com.access.log; error_log /var/log/nginx/test.com.error.log; # https://docs.djangoproject.com/en/dev/howto/static-files/#serving-static-files-in-production location /static/ { alias /www/python/apps/pyapp/static/; expires 30d; } location /media/ { alias /www/python/apps/pyapp/media/; expires 30d; } location / { uwsgi_pass unix:///tmp/pyapp.socket; include uwsgi_params; proxy_read_timeout 120; } # what to serve if upstream is not available or crashes #error_page 500 502 503 504 /media/50x.html; } </code></pre> <p>问题来了。在服务器上执行“ab”(ApacheBenchmark)时,我得到以下结果:</p> <p>nginx版本:nginx版本:nginx/1.2.6</p> <p>uwsgi版本:1.4.5</p> <pre><code>Server Software: nginx/1.0.15 Server Hostname: pycms.com Server Port: 80 Document Path: /api/nodes/mostviewed/8/?format=json Document Length: 8696 bytes Concurrency Level: 100 Time taken for tests: 41.232 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 8866000 bytes HTML transferred: 8696000 bytes Requests per second: 24.25 [#/sec] (mean) Time per request: 4123.216 [ms] (mean) Time per request: 41.232 [ms] (mean, across all concurrent requests) Transfer rate: 209.99 [Kbytes/sec] received </code></pre> <p>在500并发级别上运行时</p> <pre><code>oncurrency Level: 500 Time taken for tests: 2.175 seconds Complete requests: 1000 Failed requests: 50 (Connect: 0, Receive: 0, Length: 50, Exceptions: 0) Write errors: 0 Non-2xx responses: 950 Total transferred: 629200 bytes HTML transferred: 476300 bytes Requests per second: 459.81 [#/sec] (mean) Time per request: 1087.416 [ms] (mean) Time per request: 2.175 [ms] (mean, across all concurrent requests) Transfer rate: 282.53 [Kbytes/sec] received </code></pre> <p>如你所见。。。服务器上的所有请求都因超时错误或“客户端过早断开连接”而失败,或者:</p> <pre><code>writev(): Broken pipe [proto/uwsgi.c line 124] during GET /api/nodes/mostviewed/9/?format=json </code></pre> <p>下面是关于我的申请的更多信息: 基本上,它是反映包含所有内容的MySQL表的模型集合。在前端,我有django rest框架,它向客户端提供json内容。</p> <p>我已经安装了django profiling&django调试工具栏来查看发生了什么。关于django分析,下面是我在运行单个请求时得到的信息:</p> <pre><code>Instance wide RAM usage Partition of a set of 147315 objects. Total size = 20779408 bytes. Index Count % Size % Cumulative % Kind (class / dict of class) 0 63960 43 5726288 28 5726288 28 str 1 36887 25 3131112 15 8857400 43 tuple 2 2495 2 1500392 7 10357792 50 dict (no owner) 3 615 0 1397160 7 11754952 57 dict of module 4 1371 1 1236432 6 12991384 63 type 5 9974 7 1196880 6 14188264 68 function 6 8974 6 1076880 5 15265144 73 types.CodeType 7 1371 1 1014408 5 16279552 78 dict of type 8 2684 2 340640 2 16620192 80 list 9 382 0 328912 2 16949104 82 dict of class <607 more rows. Type e.g. '_.more' to view.> CPU Time for this request 11068 function calls (10158 primitive calls) in 0.064 CPU seconds Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.064 0.064 /usr/lib/python2.6/site-packages/django/views/generic/base.py:44(view) 1 0.000 0.000 0.064 0.064 /usr/lib/python2.6/site-packages/django/views/decorators/csrf.py:76(wrapped_view) 1 0.000 0.000 0.064 0.064 /usr/lib/python2.6/site-packages/rest_framework/views.py:359(dispatch) 1 0.000 0.000 0.064 0.064 /usr/lib/python2.6/site-packages/rest_framework/generics.py:144(get) 1 0.000 0.000 0.064 0.064 /usr/lib/python2.6/site-packages/rest_framework/mixins.py:46(list) 1 0.000 0.000 0.038 0.038 /usr/lib/python2.6/site-packages/rest_framework/serializers.py:348(data) 21/1 0.000 0.000 0.038 0.038 /usr/lib/python2.6/site-packages/rest_framework/serializers.py:273(to_native) 21/1 0.000 0.000 0.038 0.038 /usr/lib/python2.6/site-packages/rest_framework/serializers.py:190(convert_object) 11/1 0.000 0.000 0.036 0.036 /usr/lib/python2.6/site-packages/rest_framework/serializers.py:303(field_to_native) 13/11 0.000 0.000 0.033 0.003 /usr/lib/python2.6/site-packages/django/db/models/query.py:92(__iter__) 3/1 0.000 0.000 0.033 0.033 /usr/lib/python2.6/site-packages/django/db/models/query.py:77(__len__) 4 0.000 0.000 0.030 0.008 /usr/lib/python2.6/site-packages/django/db/models/sql/compiler.py:794(execute_sql) 1 0.000 0.000 0.021 0.021 /usr/lib/python2.6/site-packages/django/views/generic/list.py:33(paginate_queryset) 1 0.000 0.000 0.021 0.021 /usr/lib/python2.6/site-packages/django/core/paginator.py:35(page) 1 0.000 0.000 0.020 0.020 /usr/lib/python2.6/site-packages/django/core/paginator.py:20(validate_number) 3 0.000 0.000 0.020 0.007 /usr/lib/python2.6/site-packages/django/core/paginator.py:57(_get_num_pages) 4 0.000 0.000 0.020 0.005 /usr/lib/python2.6/site-packages/django/core/paginator.py:44(_get_count) 1 0.000 0.000 0.020 0.020 /usr/lib/python2.6/site-packages/django/db/models/query.py:340(count) 1 0.000 0.000 0.020 0.020 /usr/lib/python2.6/site-packages/django/db/models/sql/query.py:394(get_count) 1 0.000 0.000 0.020 0.020 /usr/lib/python2.6/site-packages/django/db/models/query.py:568(_prefetch_related_objects) 1 0.000 0.000 0.020 0.020 /usr/lib/python2.6/site-packages/django/db/models/query.py:1596(prefetch_related_objects) 4 0.000 0.000 0.020 0.005 /usr/lib/python2.6/site-packages/django/db/backends/util.py:36(execute) 1 0.000 0.000 0.020 0.020 /usr/lib/python2.6/site-packages/django/db/models/sql/query.py:340(get_aggregation) 5 0.000 0.000 0.020 0.004 /usr/lib64/python2.6/site-packages/MySQLdb/cursors.py:136(execute) 2 0.000 0.000 0.020 0.010 /usr/lib/python2.6/site-packages/django/db/models/query.py:1748(prefetch_one_level) 4 0.000 0.000 0.020 0.005 /usr/lib/python2.6/site-packages/django/db/backends/mysql/base.py:112(execute) 5 0.000 0.000 0.019 0.004 /usr/lib64/python2.6/site-packages/MySQLdb/cursors.py:316(_query) 60 0.000 0.000 0.018 0.000 /usr/lib/python2.6/site-packages/django/db/models/query.py:231(iterator) 5 0.012 0.002 0.015 0.003 /usr/lib64/python2.6/site-packages/MySQLdb/cursors.py:278(_do_query) 60 0.000 0.000 0.013 0.000 /usr/lib/python2.6/site-packages/django/db/models/sql/compiler.py:751(results_iter) 30 0.000 0.000 0.010 0.000 /usr/lib/python2.6/site-packages/django/db/models/manager.py:115(all) 50 0.000 0.000 0.009 0.000 /usr/lib/python2.6/site-packages/django/db/models/query.py:870(_clone) 51 0.001 0.000 0.009 0.000 /usr/lib/python2.6/site-packages/django/db/models/sql/query.py:235(clone) 4 0.000 0.000 0.009 0.002 /usr/lib/python2.6/site-packages/django/db/backends/__init__.py:302(cursor) 4 0.000 0.000 0.008 0.002 /usr/lib/python2.6/site-packages/django/db/backends/mysql/base.py:361(_cursor) 1 0.000 0.000 0.008 0.008 /usr/lib64/python2.6/site-packages/MySQLdb/__init__.py:78(Connect) 910/208 0.003 0.000 0.008 0.000 /usr/lib64/python2.6/copy.py:144(deepcopy) 22 0.000 0.000 0.007 0.000 /usr/lib/python2.6/site-packages/django/db/models/query.py:619(filter) 22 0.000 0.000 0.007 0.000 /usr/lib/python2.6/site-packages/django/db/models/query.py:633(_filter_or_exclude) 20 0.000 0.000 0.005 0.000 /usr/lib/python2.6/site-packages/django/db/models/fields/related.py:560(get_query_set) 1 0.000 0.000 0.005 0.005 /usr/lib64/python2.6/site-packages/MySQLdb/connections.py:8() </code></pre> <p>……等</p> <p>但是,django调试工具栏显示以下内容:</p> <pre><code>Resource Usage Resource Value User CPU time 149.977 msec System CPU time 119.982 msec Total CPU time 269.959 msec Elapsed time 326.291 msec Context switches 11 voluntary, 40 involuntary and 5 queries in 27.1 ms </code></pre> <p>问题是,“top”显示平均负载快速上升,而我在本地服务器和网络中的远程计算机上运行的apache基准测试显示我每秒没有处理很多请求。 怎么了?这是我在分析代码时所能达到的程度,因此如果有人能指出我在这里所做的事情,我将不胜感激。</p> <p><strong>编辑(2013年2月23日):根据Andrew Alcock的回答添加更多详细信息:</strong> 需要我注意/回答的要点是 (3) (3)我在MySQL上执行了“show global variables”,发现MySQL的配置有151个用于max_connections的设置,这已经足够服务于我为uwsgi启动的工人了。</p> <p>(3)(4)(2)我正在分析的单个请求是最重的请求。它根据django调试工具栏执行4个查询。发生的情况是,所有查询都在中运行: 分别为3.71、2.83、0.88、4.84ms。</p> <p>(4)这里你指的是内存分页?如果是,我怎么知道?</p> <p>(5)在16个工作线程上,100个并发率,1000个请求的平均负载达到~12 我在不同数量的工作线程上运行测试(并发级别为100):</p> <ol> <li>1名工人,平均负载~1.85,19请求/秒,每次请求时间:5229.520,0非2xx</li> <li>2名工人,平均负载~1.5,19请求/秒,每次请求时间:516.520,0非2xx</li> <li>4名工人,平均负载~3,16请求/秒,每次请求时间:5929.921,0非2xx</li> <li>8人,平均负载~5,18请求/秒,每次请求时间:5301.458,0非2xx</li> <li>16人,平均负载~19,15请求/秒,每次请求时间:6384.720,0非2xx</li> </ol> <p>如您所见,我们拥有的工人越多,系统的负载就越大。我可以在uwsgi的守护进程日志中看到,当我增加工作进程的数量时,响应时间(毫秒)会增加。</p> <p>在16个工作线程上,运行500个并发级别的请求uwsgi将开始记录错误:</p> <pre><code> writev(): Broken pipe [proto/uwsgi.c line 124] </code></pre> <p>负载也增加到~10。测试不需要花费太多时间,因为非2xx响应是1000个响应中的923个,这就是为什么这里的响应非常快,因为它几乎是空的。这也是对你在总结中第4点的答复。</p> <p>假设我在这里所面临的是基于i/O和网络的操作系统延迟,那么建议采取什么措施来扩大这种延迟?新硬件?更大的服务器?</p> <p>谢谢</p>
0 条评论
分类:
Python问答
请先
登录
后评论
默认排序
时间排序
1 个回答
匿名
1天前
擅长:python、mysql、java
<p>添加更多的工作线程和获得更少的r/s意味着您的请求“是纯CPU”,并且没有IO等待,其他工作线程可以用来服务另一个请求。</p> <p>如果您想扩展,您将需要使用另一台cpu更多(或更快)的服务器</p> <p>然而,这是一个综合测试,您得到的r/s数是您正在测试的确切请求的上限,一旦投入生产,就会有更多的变量影响性能。</p>
请先
登录
后评论
针对此问题:
更多的回答
关注
89
关注
收藏
1
收藏,
216
浏览
网友 提问于 2天前
相关Python问题
尝试加密和解密文本文件(Python)
2 回答
尝试加密文本文件会导致类型错误
10 回答
尝试加密时溢出错误
4 回答
尝试加载.plist文件时出现“无此类文件或目录”错误
10 回答
尝试加载“blog.templatetags.blog_标记”时引发ImportError:没有名为“markdown”的模块
6 回答
尝试加载“menus.templatetags.menus\u标记”时引发ImportError:无法从“menus.models”导入名称“menus”
5 回答
尝试加载5GB文本fi时发生Python内存错误
8 回答
尝试加载5GB文本文件时出现MemoryError
5 回答
尝试加载Django模板时出现渲染错误
3 回答
尝试加载exi时出错
6 回答
尝试加载imag时使用精灵表失败
2 回答
尝试加载json.Python时出错
2 回答
尝试加载json文件时遇到此错误:json.decoder.jsondeCoderror:预期值:第1行第1列(char 0)
2 回答
尝试加载keras和tensorflow,我仍然得到ModuleNotFoundError:在我的Anaconda环境中没有名为'tensorflow_core.estimator'的模块
7 回答
尝试加载Librosa的示例fi时出现“找不到文件”错误
7 回答
尝试加载matplotlib时发生iPython笔记本错误
10 回答
尝试加载MP3时出现PyGame错误
3 回答
尝试加载pickled matplotlib figu时发生AttributeError
9 回答
尝试加载pickle文件时出错
1 回答
尝试加载Pytoch模型时遇到问题:“在模块中找不到标识”
1 回答