You Developed an ML Model with AI Platform, and You

question 14

Multiple Choice

You developed an ML model with AI Platform, and you want to move it to production. You serve a few thousand queries per second and are experiencing latency issues. Incoming requests are served by a load balancer that distributes them across multiple Kubeflow CPU-only pods running on Google Kubernetes Engine (GKE) . Your goal is to improve the serving latency without changing the underlying infrastructure. What should you do?

• A) Significantly increase the max_batch_size TensorFlow Serving parameter. Significantly increase the max_batch_size TensorFlow Serving parameter.
• B) Switch to the tensorflow-model-server-universal version of TensorFlow Serving.
• C) Significantly increase the max_enqueued_batches TensorFlow Serving parameter. max_enqueued_batches
• D) Recompile TensorFlow Serving using the source to support CPU-specific optimizations. Instruct GKE to choose an appropriate baseline minimum CPU platform for serving nodes.

---

Definitions:

U.S. Department Of Defense

The federal department charged with coordinating and supervising all agencies and functions of the government relating directly to national security and the United States armed forces.

Internet

A global system of interconnected computer networks that use the Internet protocol suite to communicate between networks and devices, facilitating information sharing and communication.

Offshoring

The relocation of a business process or operation from one country to another, typically to leverage lower labor costs.

Globalization

Globalization is the process of increased interconnectedness and interdependence among countries, mainly due to trade, investment, and technology.