Видео

what we use in production is when glue job put data in raw s3 bucket it will create an AWS SNS notification which is subscribed by SQS then with the help of queue we trigger lambda :)

This is very informative. Thank you!

can we combine batch processing with step function?

Can this be used to read few columns from s3 bucket?

Thank you for creating such wonderful videos.Very informative and great learning for us.

Like it

Good explanation but you take long time explaining the small topic and dragging can you please work on it!!!

How does job bookmark scale on massive data set ?

Copy job works in large files ??

very informative sir... :)

you better use a headset or earphone while speaking.. otherwise the session is very good.

Thank you for explain in detail

I am literally searching for this. Thank u

very good presentation

whenever i reach to notebook instance step: resourcelimitexceeds message comes

Its very good....question: If test data only 20 records even then model takes 30 mins to train,why? In your eaxmple 2000 data taking 1 +hour

Thankyou 😇

Thanks for this video. It helped me a lot. Question: How do you deal with timestamp(3) in AWS Athena and timestamp(6) in Iceberg?

Connecting to 'endpoint' with client ID 'client-id' Traceback (most recent call last): File "c:\Users\DELL\Desktop\Certificates\aws3.py", line 34, in <module> connect_future.result() File "C:\Users\DELL\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\_base.py", line 456, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "C:\Users\DELL\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\_base.py", line 401, in __get_result raise self._exception awscrt.exceptions.AwsCrtError: AWS_ERROR_MQTT_UNEXPECTED_HANGUP: The connection was closed unexpectedly. I getting some error could you please check this error @AWS Tutorials

@AWSTutorialsOnline, Appreciate your good work. AWS glue has evolved so much now, how can we in-corporate data quality checks to the pipelines and send email notifications to the users with dq fail results such as rules_succeeded, rules_skipped, rules_failed and publish the data to a quicksight dashboard. Do we still need step-functions ? Any thoughts / suggestions please.

How can we display glue workflow details in a grafana dashboard?

sir, have a qq, what if the 1st record in side the file in S3 doesnt have any header record in case of a CSV file. ? does crawler still be able to derrive the data type of the columns based on data ?

Hi. Am I supposed to bring 20 tables if my query uses then to create the SQL query using Glue visual ETL?

Demo part is not good. things are not properly explained. Just reading, not shown up to how to create them up. please focus on practical part instead of theory.

Is that cost effective to have a single job running multiple times or multiple job runs once?

Can you comment on getting data from SaaS applications which only provide APIs, and something getting data of large volume may take 5-6hours. Is glue the right approach?

Awesome explanation

Nice tutorial,Thanks. can it run in sequence? i want to run the jobs with different parameter, but i want the second job run after the first one is finished. Like a queue. Or we must set the max concurrent to 1 and handle the retry ourself if max concurrent error occurred?

Great Great tutorial! I have one request, From where i can download sales and customer data set ?

Great content as always!!

unbox command doesn't work for me?

You Are Awesome 🙂

Well detailed information for beginners

Can you please make a video on delta files to achieve scd type 1, because in this scenario it was full file ,but i want to process on incremental files

Can you show an example of how to achieve the same in Glue PythonShell Job

Hi Sir, Is it possible to enable job bookmark for concurrent job run but single script with step function?

Very nicely explained..

Thanks for the video, Can you describe how this job was run behind the scenes and any way to control the parquet file size per block size?

thank you sir

thank you sir

This is awesome, glad i found this channel.

This is really helpful, i just started to use AWS Glue recently. Hats off to you Sir!

Short sweet information sir ❤

Great video, i was already going to resolve that with a lambda, so more easy with glue data quality, thank you

Sir basically i have 2 endpoints in appsync api So i am writing testcases for it. Basically whatever query we add to endpoints shoudld be dynamic and also it should form like schema given.. How can we add validation here for query before hitting api via python code?

How we can configure the Aws glue job logs via cloud watch and get the notification of the error and successful runs in the mail in detailed?

Can I know what is benefit of using kendra over using bedrock fms to build such product search application in cost base,I want to know charges info , just specify which charges much and why

Good job - helpful content

What to choose under "New" option if I will be doing Scala code in Spark instead of python?