origo数据平台SDK
okdata-sdk的Python项目详细描述
okdata-sdk:Origo数据平台的Python SDK
^{pip install okdata-sdk
配置
当调用与Origo dataplatformapi交互的任何类，并且没有传递给构造函数的配置参数时，配置对象将 基于环境变量自动为您创建
环境变量
默认情况下，将从当前环境中获取配置。 如果未按以下顺序设置特定的身份验证配置，则会自动解析凭据：
- Client Credentials：如果您已将Client_id/Client_secret添加到配置中。或者如果你用
等效环境变量：
OKDATA_CLIENT_ID/OKDATA_CLIENT_SECRET。在 - Username And Password：如果您在配置中添加了Username/Password。或者如果你用
等效环境变量：
OKDATA_USERNAME/OKDATA_PASSWORD。在
# keycloak user
export OKDATA_USERNAME=my-user
# keycloak password for OKDATA_USERNAME
export OKDATA_PASSWORD=my-password
# keycloak client
export OKDATA_CLIENT_ID=my-machine-client
# keycloak secret for OKDATA_CLIENT_ID
export OKDATA_CLIENT_SECRET=some-generated-secure-string
# overrides default environment (dev), but will be trumped by --env=<environment> on the commandline
export OKDATA_ENVIRONMENT=dev|prod
# If you are sending events and have been assigned a API key
export OKDATA_API_KEY=your-api-key
获取凭据：
username/password 与奥斯陆市的Active Directory同步，因此任何具有关联的用户都可以
使用他们的个人帐户访问SDK。在
关于client credentials请联系数据平台团队。dataplattform[at]oslo.kommune.no
TODO:命名配置文件
如果环境变量不可用，系统将尝试从~/.okdata/configuration中的默认配置文件加载
使用
目录：
- Upload data
- Sending events
- Create and manage event streams
- Creating datasets with versions and editions
上传数据
上载数据时，您需要引用您拥有的现有数据集、版本和版本。 如果这些不存在，那么您可以自己创建它们。这可以实现using the sdk， 或者你可以用我们的command line interface。在
^{pr2}$upload方法返回的trace_id可用于“跟踪”上载过程中涉及的步骤：
fromokdata.sdk.statusimportStatus...status=Status(config=okdata_config)trace_events=status.get_status(trace_id)print(trace_events)# [# {# "trace_id": "my-dataset-1a2bc345-6789-1234-567d-8912ef34a567",# "trace_status": "STARTED",# "trace_event_id": "1a2b3cd4-eef5-6aa7-bccd-e889912334f5",# "trace_event_status": "OK",# "component": "data-uploader",# ...# },# {# "trace_id": "my-dataset-1a2bc345-6789-1234-567d-8912ef34a567",# "trace_status": "CONTINUE",# ...# },# {# "trace_id": "my-dataset-1a2bc345-6789-1234-567d-8912ef34a567",# "trace_event_id": "1aa2b345-678c-9de1-f2a3-4566bcd78912",# "trace_status": "FINISHED",# "trace_event_status": "OK",# ...# }# ]
下载数据
下载数据时，您需要引用您拥有的现有数据集、版本和版本。 如果这些不存在，那么您可以自己创建它们。这可以实现using the sdk， 或者你可以用我们的command line interface。在
fromokdata.sdk.data.downloadimportDownloadfromokdata.sdk.configimportConfigokdata_config=Config(env="dev")# If necessary you can override default config valuesokdata_config.config["cacheCredentials"]=Falsedata_downloader=Download(config=okdata_config)dataset_id="your-dataset-id"version="1"edition="latest"# Downloading a fileres1=data_downloader.download(dataset_id,version,edition,"my/preferred/output/path")print(res1)# {# "downloaded_files": ["my/preferred/output/path/file_name.csv"]# }
发送事件
为了开始发送事件，您需要访问事件流。如果这样的事件流已经 在适当的地方你可以去。如果没有，可以通过using the sdk创建一个， 或者通过using our command line interface。在
fromokdata.sdk.event.post_eventimportPostEventfromokdata.sdk.configimportConfigokdata_config=Config()# If necessary you can override default config valuesokdata_config.config["cacheCredentials"]=Trueevent_poster=PostEvent(config=okdata_config)dataset_id="some-dataset-id"version="1"event={"foo":"bar"}res=event_poster.post_event(event,dataset_id,version)# res:# {'message': 'Ok'}# Method also supports list of dictionariesevent_list=[{"foo":"bar"},{"foo":"bar"}]res2=event_poster.post_event(event_list,dataset_id,version)# res2:# {'message': 'Ok'}
创建和管理事件流
为了创建事件流，您需要定义一个数据集和一个版本， 除非这些已经存在。定义数据集和版本可以是 实现了{a13}， 或者你可以用我们的command line interface。 创建事件流不需要定义版本。在
fromokdata.sdk.event.event_stream_clientimportEventStreamClient# Using default configuration for dev-environmentevent_stream_client=EventStreamClient(env="dev")dataset_id="some-dataset-id"version="1"# Creating a new event stream:create_response=event_stream_client.create_event_stream(dataset_id,version)# create_response:# {'message': 'Accepted'}# Getting info about the event streamevent_stream_info=event_stream_client.get_event_stream_info(dataset_id,version)# event_stream_info:# { 'createdAt': '2020-01-29T07:02:32.598520+00:00',# 'createdBy': 'jd',# 'id': 'test-stream-manager/1',# 'status': 'CREATE_IN_PROGRESS'# }# Note! You must wait until the event stream has status=ACTIVE# before you can successfully send events to the stream# Deleting the event streamdelete_response=event_stream_client.delete_event_stream(dataset_id,version)# delete_response:# {'message': 'Delete initiated'}
使用版本和版本创建数据集
fromokdata.sdk.data.datasetimportDatasetfromokdata.sdk.configimportConfigokdata_config=Config()# If necessary you can override default valuesokdata_config.config["cacheCredentials"]=False# Create a new datasetdataset=Dataset(config=okdata_config)dataset_metadata={"title":"Precise Descriptive Title","description":"Describe your dataset here","keywords":["some-keyword"],"accessRights":"public","confidentiality":"green","objective":"Exemplify how to create a new dataset","contactPoint":{"name":"Your name","email":"your_email@domain.com","phone":"999555111"},"publisher":"name of organization or person responsible for publishing the data"}new_dataset=dataset.create_dataset(data=dataset_metadata)# new_dataset:# { 'Id': 'precise-descriptive-title',# 'Type': 'Dataset',# '_links': {'self': {'href': '/datasets/precise-descriptive-title'}},# 'accessRights': 'public',# 'confidentiality': 'green',# 'contactPoint': { 'email': 'your_email@domain.com',# 'name': 'Your name',# 'phone': '999555111'},# 'description': 'Describe your dataset here',# 'keywords': ['some-keyword'],# 'objective': 'Exemplify how to create a new dataset',# 'publisher': 'name of organization or person responsible for publishing the '# 'data',# 'title': 'Precise Descriptive Title'}# create version for new dataset:version_data={"version":"1"}new_version=dataset.create_version(new_dataset["Id"],data=version_data)# new_version:# { 'Id': 'precise-descriptive-title/1',# 'Type': 'Version',# '_links': { 'self': { 'href': '/datasets/precise-descriptive-title/versions/1'}},# 'version': '1'}# create edition for new_dataset/new_version:importdatetime# Note! edition-field must be ISO 8601 with utc offsetedition_data={"edition":str(datetime.datetime.utcnow().replace(tzinfo=datetime.timezone.utc).isoformat()),"description":"My edition description","startTime":"2019-01-01","endTime":"2019-12-31"}new_edition=dataset.create_edition(new_dataset["Id"],new_version["version"],data=edition_data)# new_edition# { 'Id': 'precise-descriptive-title/1/20200115T130439',# 'Type': 'Edition',# '_links': { 'self': { 'href': '/datasets/precise-descriptive-title/versions/1/editions/20200115T130439'}},# 'description': 'My edition description',# 'edition': '2020-01-15T13:04:39.041778+00:00',# 'endTime': '2019-12-31',# 'startTime': '2019-01-01'}
- 项目
标签：
欢迎加入QQ群-->： 979659372
