DATABRICKS - DATA ENGINEER ASSOCIATE EXAM WITH GUARANTEED ACCURATE ANSWERS
0 view 0 purchase
Course
DATABRICKS - DATA ENGINEER ASSOCIATE
Institution
DATABRICKS - DATA ENGINEER ASSOCIATE
DATABRICKS - DATA ENGINEER ASSOCIATE EXAM
DATABRICKS - DATA ENGINEER ASSOCIATE EXAM WITH GUARANTEED ACCURATE ANSWERS
You were asked to create a table that can store the below data, <orderTime> is a timestamp but the finance team when they query this data normally prefer the <orderTi...
DATABRICKS - DATA ENGINEER
ASSOCIATE EXAM
DATABRICKS - DATA ENGINEER ASSOCIATE EXAM WITH GUARANTEED ACCURATE
ANSWERS
You were asked to create a table that can store the below data,
<orderTime> is a timestamp but the finance team when they query this
data normally prefer the <orderTime> in date format, you would like to
create a calculated column that can convert the <orderTime> column
timestamp datatype to date and store it, fill in the blank to complete the
DDL.
CREATE TABLE orders (
orderId int,
orderTime timestamp,
orderdate date _____________________________________________ ,
units int)
,A. AS DEFAULT (CAST(orderTime as DATE))
B. GENERATED ALWAYS AS (CAST(orderTime as DATE))
C. GENERATED DEFAULT AS (CAST(orderTime as DATE))
D. AS (CAST(orderTime as DATE))
E. Delta lake does not support calculated columns, value should be
inserted into the table as part of the ingestion process - ACCURATE
ANSWERS✔✔ B. GENERATED ALWAYS AS (CAST(orderTime as
DATE))
Explanation
The answer is, GENERATED ALWAYS AS (CAST(orderTime as
DATE))
Delta Lake supports generated columns which are a special type of
columns whose values are automatically generated based on a user-
specified function over other columns in the Delta table. When you write
to a table with generated columns and you do not explicitly provide
values for them, Delta Lake automatically computes the values.
Note: Databricks also supports partitioning using generated column
,The data engineering team noticed that one of the job fails randomly as a
result of using spot instances, what feature in Jobs/Tasks can be used to
address this issue so the job is more stable when using spot instances?
A. Use Databrick REST API to monitor and restart the job
B. Use Jobs runs, active runs UI section to monitor and restart the job
C. Add second task and add a check condition to rerun the first task if it
fails
D. Restart the job cluster, job automatically restarts
E. Add a retry policy to the task - ACCURATE ANSWERS✔✔ E. Add a
retry policy to the task
The answer is, Add a retry policy to the task
Tasks in Jobs support Retry Policy, which can be used to retry a failed
tasks, especially when using spot instance it is common to have failed
executors or driver.
What is the main difference between AUTO LOADER and COPY
INTO?
A. COPY INTO supports schema evolution.
B. AUTO LOADER supports schema evolution.
C. COPY INTO supports file notification when performing incremental
loads.
, D. AUTO LOADER supports reading data from Apache Kafka
E, AUTO LOADER Supports file notification when performing
incremental loads. - ACCURATE ANSWERS✔✔ E, AUTO LOADER
Supports file notification when performing incremental loads.
Explanation
Auto loader supports both directory listing and file notification but
COPY INTO only supports directory listing.
Auto loader file notification will automatically set up a notification
service and queue service that subscribe to file events from the input
directory in cloud object storage like Azure blob storage or S3. File
notification mode is more performant and scalable for large input
directories or a high volume of files.
Auto Loader and Cloud Storage Integration
Auto Loader supports a couple of ways to ingest data incrementally
Directory listing - List Directory and maintain the state in RocksDB,
supports incremental file listing
File notification - Uses a trigger+queue to store the file notification
which can be later used to retrieve the file, unlike Directory listing File
notification can scale up to millions of files per day.
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller GRADEUNITS. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $16.49. You're not tied to anything after your purchase.
