使⽤streamlit和polyaxon将机器学习应⽤程序部署到
kubernetes
本指南的学习⽬标(Learning Goals of this guide)
Brief introduction to containers, , , and .
容器, , 和简要介绍。
Create a Kubernetes cluster and deploy with .
创建⼀个Kubernetes集群并使⽤部署 。
How to explore datats on a running on a Kubernetes cluster.
如何在Kubernetes集群上运⾏的上浏览数据集。
How to train multiple versions of a machine learning model using on Kubernetes.
如何使⽤上的Polyaxon训练机器学习模型的多个版本。
How to save a machine learning model.
如何保存机器学习模型。
小清新风景图片How to analyze the models using UI.
助学贷款申请如何使⽤ UI分析模型。
How to expo the model with a ur interface using and make new predictions.
如何使⽤通过⽤户界⾯公开模型并做出新的预测。
本指南所需的⼯具 (Tools Required for this guide)
什么是容器?(What is a container?)
A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another. is a tool designed to make it easier to create, deploy, and run applications by using containers.
容器是打包代码及其所有依赖项的软件的标准单元,因此该应⽤程序可以从⼀个计算环境快速可靠地
运⾏到另⼀个计算环境。 是⼀种⼯具,旨在使使⽤容器更容易创建,部署和运⾏应⽤程序。
In our guide we will u containers to package our code and dependencies and easily deploy them on Kubernetes.
在我们的指南中,我们将使⽤容器打包我们的代码和依赖项,并轻松地将它们部署在Kubernetes上。
什么是Kubernetes? (What is Kubernetes?)
is a powerful open-source distributed system for managing containerized applications. In simple words, Kubernetes is a system for running and orchestrating containerized applications across a cluster of machines. It is a platform designed to completely manage the life cycle of containerized applications.
是⽤于管理容器化应⽤程序的功能强⼤的开源分布式系统。 简单来说就是Kubernetes 是⼀个⽤于跨机器集群运⾏和协调容器化应⽤程序的系统。 它是⼀个旨在完全管理容器化应⽤程序⽣命周期的平台。
Why should I u Kubernetes.
我为什么要使⽤Kubernetes。
Load Balancing: Automatically distributes the load between containers.
负载平衡:⾃动在容器之间分配负载。
Scaling: Automatically scale up or down by adding or removing containers when demand changes such as peak hours, weekends and holidays.
缩放:在需求变化(例如⾼峰时间,周末和节假⽇)时,通过添加或删除容器来⾃动缩放。
Storage: Keeps storage consistent with multiple instances of an application.
存储:使存储与应⽤程序的多个实例保持⼀致。
Self-healing Automatically restarts containers that fail and kills containers that don’t respond to your ur-defined health check.
⾃我修复功能⾃动重新启动失败的容器并杀死不响应⽤户定义的运⾏状况检查的容器。
Automated Rollouts you can automate Kubernetes to create new containers for your deployment, remove existing containers and adopt all of their resources to the new container.
⾃动化推出,您可以⾃动化Kubernetes来为您的部署创建新容器,删除现有容器并将其所有资源⽤于新容器。
什么是Streamlit? (What is Streamlit?)
is an open-source framework to create an interactive, beautiful visualization app. All in python!
是⼀个开放源代码框架,⽤于创建交互式,美观的可视化应⽤程序。 全部在python中!
Streamlit provides many uful features that can be very helpful in making visualizations for data-driven projects.
Streamlit提供了许多有⽤的功能,这些功能对于使数据驱动的项⽬可视化⾮常有⽤。
Example of Face-GAN explorer using Streamlit
使⽤Streamlit的Face-GAN资源管理器⽰例
model using 演⽰了 . 模型。
Why should I u Streamlit?
为什么要使⽤Streamlit?
Simple and easy way to create an interactive ur interface
创建交互式⽤户界⾯的简便⽅法
Requires zero development experience
需要零开发经验
It’s fun making u of different function in your data-driven projects :)
在数据驱动的项⽬中使⽤不同的功能很有趣:)
Comprehensive documentation
综合⽂件
什么是Polyaxon?(What is Polyaxon?)
is an open-source cloud native machine learning platform, that provides simple interfaces to train, monitor, and manage models.
励志名言英语是⼀个开源的云原⽣机器学习平台,它提供了简单的界⾯来训练,监视和管理模型。
runs on top of Kubernetes to allow scaling up and down of your cluster’s resources, and provides tools to automate the process of experimentation, while tracking information about models, configurations, parameters, and code.
在运⾏,以允许扩展和缩减集群的资源,并提供⼯具来⾃动执⾏实验过程,同时跟踪有关模型,配置,参数和代码的信息。
Why should I u Polyaxon?
钦州电视台为什么要使⽤Polyaxon?
Automatically track key model metrics, hyperparameters, visualizations, artifacts and resources, and version control code and data.
⾃动跟踪关键模型指标,超参数,可视化,⼯件和资源以及版本控制代码和数据。
Maximize the usage of your cluster by scheduling jobs and experiments via the CLI, dashboard, SDKs, or REST API.
通过CLI,仪表板,SDK或REST API安排作业和实验,从⽽最⼤程度地利⽤群集。
泰式spa
U optimization algorithms to effectively run parallel experiments and find the best model.
使⽤优化算法有效地运⾏并⾏实验并找到最佳模型。
Visualize, arch, and compare experiment results, hyperparams, training data and source code versions, so you can quickly analyze what worked and what didn’t.
可视化,搜索和⽐较实验结果,超参数,训练数据和源代码版本,因此您可以快速分析哪些有效,哪些⽆效。
红旗谱读后感
Consistently develop, validate, deliver, and monitor models to create a competitive advantage.
⼀致地开发,验证,交付和监视模型以创建竞争优势。
解释的英文Scale your resources as needed, and run jobs and experiments on any platform (AWS, Microsoft Azure, Google Cloud Platform, and on-premis hardware).
根据需要扩展资源,并在任何平台(AWS,Microsoft Azure,Google Cloud Platform和本地硬件)上运⾏作业和实验。
什么是头盔? (What is Helm?)
is the package manager for Kubernetes, it allows us to deploy and manage the life cycle of cloud native projects like Polyaxon.
是Kubernetes的软件包经理,它使我们能够部署和管理Polyaxon等云原⽣项⽬的⽣命周期。
Azure Kubernetes服务 (Azure Kubernetes Service)
In this tutorial we will be using , a fully managed Kubernetes rvice on Azure. If you do not have an account with Azure, you can sign-up for a free account.
在本教程中,我们将使⽤ ,这是Azure上完全托管的Kubernetes服务。 如果您没有使⽤Azure的帐户,则可以注册⼀个免费帐户。
In future posts, we will provide similar instructions of running this guide on Google Cloud Platform (GKE), AWS (EKS), and a local cluster with Minikube.
在以后的⽂章中,我们将提供在Google Cloud Platform (GKE) , AWS (EKS)和Minikube本地集群上运⾏此指南的类似说明。
设置⼯作区 (Setting the workspace)
The purpo of this tutorial is to get hands-on experience of running machine learning experimentation and deployment on Kubernetes. Let’s get started by creating our workspace.
本教程的⽬的是获得在Kubernetes上运⾏机器学习实验和部署的实践经验。 让我们开始创建⼯作区。
第1步—使⽤AKS部署Kubernetes集群 (Step 1 — Deploy a Kubernetes cluster with AKS)
Let’s create a simple with a single node:
让我们具有单个节点的上创建⼀个简单的:
az aks create --resource-group myResourceGroup --name streamlit-polyaxon --node-count 1 --enable-addons monitoring --generate-ssh-keys长安一片月下一句
To make sure you are on the right cluster you can execute the command
为了确保您在正确的集群上,可以执⾏以下命令
az aks get-credentials --resource-group
第2步-安装头盔(Step 2 — Install Helm)
Install on your local machine to be able to manage Polyaxon as well as other cloud native projects that you might want to run on Kubernetes.
在本地计算机上安装 ,以便能够管理Polyaxon以及您可能希望在Kubernetes上运⾏的其他云本机项⽬。
curl -fsSL -o get_helm.sh chmod 700 get_helm.sh./get_helm.sh
第3步-将Polyaxon图表添加到Helm (Step 3— Add Polyaxon Charts to Helm)
helm repo add polyaxon
第4步-安装Polyaxon CLI (Step 4— Install Polyaxon CLI)
pip install -U polyaxon
第5步—将Polyaxon部署到Kubernetes (Step 5—Deploy Polyaxon to Kubernetes)
polyaxon admin deploy
第6步-等待部署达到就绪状态 (Step 6—Wait for the deployments to reach the ready state) kubectl ge
t deployment -n polyaxon -w
This should take about 3 min:
这⼤约需要3分钟:
NAME READY UP-TO-DATE AVAILABLE AGE
polyaxon-polyaxon-api 1/1 1 1 3m17s
polyaxon-polyaxon-gateway 1/1 1 1 3m17s
polyaxon-polyaxon-operator 1/1 1 1 3m17s
polyaxon-polyaxon-streams 1/1 1 1 3m17s
第7步-公开Polyaxon API和UI (Step 7— Expo Polyaxon API and UI)
Polyaxon provides a simple command to expo the dashboard and the API in a cure way on your localhost: Polyaxon提供了⼀个简单的命令,可以在您的本地主机上以安全的⽅式公开仪表板和API:
polyaxon port-forward
第8步-在Polyaxon上创建⼀个项⽬ (Step 8— Create a project on Polyaxon)
In a different terminal ssion than the one ud for exposing the dashboard, run:
在不同于⽤于显⽰仪表板的终端会话中,运⾏:
polyaxon project create --name=streamlit-app
You should e:
您应该看到:
Project `streamlit-app` was created successfully.
You can view this project on Polyaxon UI:
Streamlit project on Polyaxon
关于Polyaxon的Streamlit项⽬
Now we can move to the next ction: training and analyzing a model.
现在,我们可以进⼊下⼀部分:训练和分析模型。