You've Migrated a Hadoop Job from an On-Prem Cluster to Dataproc

question 133

Multiple Choice

You've migrated a Hadoop job from an on-prem cluster to dataproc and GCS. Your Spark job is a complicated analytical workload that consists of many shuffing operations and initial data are parquet files (on average 200-400 MB size each) . You see some degradation in performance after the migration to Dataproc, so you'd like to optimize for it. You need to keep in mind that your organization is very cost-sensitive, so you'd like to continue using Dataproc on preemptibles (with 2 non-preemptible workers only) for this workload. What should you do?

• A) Increase the size of your parquet files to ensure them to be 1 GB minimum.
• B) Switch to TFRecords formats (appr. 200MB per file) instead of parquet files.
• C) Switch from HDDs to SSDs, copy initial data from GCS to HDFS, run the Spark job and copy results back to GCS.
• D) Switch from HDDs to SSDs, override the preemptible VMs configuration to increase the boot disk size.

Identify the process and importance of environmental scanning in adapting to market changes.

Differentiate between various environmental forces (social, economic, technological, competitive, and regulatory) affecting marketing strategies.

Analyze the influence of social forces on market trends and strategies.

Examine the role of technological advancements in shaping consumer behaviors and expectations.

---

Definitions:

Short-Term Scheduling

The process of organizing and optimizing operations or tasks in the near term, focusing on efficient allocation of resources to meet immediate production or service goals.

Assigns Jobs

The act of allocating or designating work tasks to individuals or groups based on criteria such as skills, availability, or strategic importance.

Work Centres

Specific locations, machines, or groups of machines within a manufacturing facility where distinct types of processes are performed.

Assignment Method

The assignment method is a mathematical technique used for allocating resources or tasks to units in an efficient manner, often used in operations research.