具有记录聚合的aws kinisis流的生产者
kinesis-producer的Python项目详细描述
带有记录聚合的aws kinisis流的python producer。
它使用boto 3，并在python 2.7和3.4/3.5上进行了测试。
安装
pip install ‘kinesis_producer < 1’
注意:kinisis producer use semver:您应该始终冻结少校 因为它可能意味着破坏api。
用法
发送总计高达100kb、200ms并用“\n”连接的记录：
fromkinesis_producerimportKinesisProducerconfig=dict(aws_region='us-east-1',buffer_size_limit=100000,buffer_time_limit=0.2,kinesis_concurrency=1,kinesis_max_retries=10,record_delimiter='\n',stream_name='KINESIS_STREAM_NAME',)k=KinesisProducer(config=config)forrecordinrecords:k.send(record)k.close()k.join()
配置
| aws_region: | AWS region for Kinesis calls (like us-east-1) |
|---|---|
| buffer_size_limit: | |
| Approximative size limit for record aggregation (in bytes) | |
| buffer_time_limit: | |
| Approximative time limit for record aggregation (in seconds) | |
| kinesis_concurrency: | |
| Set the concurrency level for Kinesis calls. Set to 1 for no concurrency. Set to 2 and more to use a thread pool. | |
| kinesis_max_retries: | |
| Number of Kinesis put_records call attempt before giving up. This number should be between 4 and 10 if you want to handle temporary ProvisionedThroughputExceeded errors. | |
| record_delimiter: | |
| Delimiter for record aggregation | |
| stream_name: | Name of the Kinesis Stream |
动觉重试
为ProvisionedThroughputExceeded错误重试动态播放调用 只有。重试使用指数退避逻辑（0.1s、0.2s、0.4s、0.8s， 1.60秒、3.20秒、6.40秒、12.80秒、25.60秒、51.20秒、102.40秒……
版权和许可
在麻省理工学院的许可下发布。
欢迎加入QQ群-->： 979659372
