# ![@rajasdataengineering7585 Avatar](https://lunarcrush.com/gi/w:26/cr:youtube::UCv4yD1y59pf69GurCYceT9Q.png) @rajasdataengineering7585 Raja's Data Engineering

Raja's Data Engineering posts on YouTube about databricks, azure, engineering, delta the most. They currently have [------] followers and [--] posts still getting attention that total [---] engagements in the last [--] hours.

### Engagements: [---] [#](/creator/youtube::UCv4yD1y59pf69GurCYceT9Q/interactions)
![Engagements Line Chart](https://lunarcrush.com/gi/w:600/cr:youtube::UCv4yD1y59pf69GurCYceT9Q/c:line/m:interactions.svg)

- [--] Week [-----] +28%
- [--] Month [------] +31%
- [--] Months [------] -44%
- [--] Year [-------] -18%

### Mentions: [--] [#](/creator/youtube::UCv4yD1y59pf69GurCYceT9Q/posts_active)
![Mentions Line Chart](https://lunarcrush.com/gi/w:600/cr:youtube::UCv4yD1y59pf69GurCYceT9Q/c:line/m:posts_active.svg)

- [--] Month [--] -29%
- [--] Months [--] -36%
- [--] Year [--] +150%

### Followers: [------] [#](/creator/youtube::UCv4yD1y59pf69GurCYceT9Q/followers)
![Followers Line Chart](https://lunarcrush.com/gi/w:600/cr:youtube::UCv4yD1y59pf69GurCYceT9Q/c:line/m:followers.svg)

- [--] Week [------] +0.26%
- [--] Month [------] +1.10%
- [--] Months [------] +7.30%
- [--] Year [------] +25%

### CreatorRank: [---------] [#](/creator/youtube::UCv4yD1y59pf69GurCYceT9Q/influencer_rank)
![CreatorRank Line Chart](https://lunarcrush.com/gi/w:600/cr:youtube::UCv4yD1y59pf69GurCYceT9Q/c:line/m:influencer_rank.svg)

### Social Influence

**Social category influence**
[technology brands](/list/technology-brands)  [social networks](/list/social-networks) 

**Social topic influence**
[databricks](/topic/databricks) #25, [azure](/topic/azure), [engineering](/topic/engineering) #2278, [delta](/topic/delta), [more than](/topic/more-than), [beginner](/topic/beginner), [how to](/topic/how-to), [the most](/topic/the-most), [common](/topic/common), [trigger](/topic/trigger)

**Top assets mentioned**
[Spark (SPK)](/topic/spark)
### Top Social Posts
Top posts by engagements in the last [--] hours

"01. Databricks: Spark Architecture & Internal Working Mechanism #SparkArchitecture #DatabricksArchitecture #Masterslave #DriverWorker #SparkExecutor #Spark Memory management #Sparkjobs #SparkRDD #Databricks #DatabricksTutorial #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure databricks notebook tutorial databricks delta lake databricks azure tutorial Databricks Tutorial for beginners azure Databricks tutorial databricks tutorial databricks"  
[YouTube Link](https://youtube.com/watch?v=4JP0XqsjwCI)  2021-07-10T15:49Z 38.4K followers, 434.6K engagements


"02. Databricks PySpark: RDD Dataframe and Dataset #Databricks #DatabricksTutorial #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure databricks notebook tutorial databricks delta lake databricks azure tutorial Databricks Tutorial for beginners azure Databricks tutorial databricks tutorial databricks community edition databricks community edition cluster creation databricks community edition tutorial databricks community edition pyspark databricks"  
[YouTube Link](https://youtube.com/watch?v=g4T25_4HGM0)  2021-07-05T16:06Z 38.4K followers, 112.5K engagements


"96. Databricks Pyspark Real Time Scenario Schema Comparison Azure Databricks Learning: Schema Comparison ============================================ How to compare schemas of different dataframes and make it same through automated way Schema related operations are very common in Databricks development. Have explained different real time scenarios of schema related operations in this video To get through understanding of this concept please watch this video #DatabricksSchemaComparison #DatabricksStructType#DatabricksMapType#PysparkStructType#PysparkMapType"  
[YouTube Link](https://youtube.com/watch?v=BtUFleFkXMM)  2023-02-03T14:15Z 38.4K followers, 11.1K engagements


"34. Databricks - Spark: Data Skew Optimization #DataSkew #Bigdata-Dataskew #BigdataOptimization #AdaptiveQueryExecution #AQE #DatabricksDataskew #SparkSalting #Salting #DatabricksSalting #SkewHint #SparkSkewhint #DatabricksOptimization#pysparkOptimization #sparkOptmimization #SparkPerformanceOptimization #SparkPerformance #DatabricksPerformanceImprovement#Databricks #DatabricksTutorial #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure databricks"  
[YouTube Link](https://youtube.com/watch?v=EQhldyLWPwI)  2021-12-10T15:39Z 37K followers, 41K engagements


"22. Databricks Spark Performance Optimization Repartition vs Coalesce #DatabricksPerformance #SparkPerformance #PerformanceOptimization #DatabricksPerformanceImprovement #Repartition #Coalesce #Databricks #DatabricksTutorial #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure databricks notebook tutorial databricks delta lake databricks azure tutorial Databricks Tutorial for beginners azure Databricks tutorial databricks tutorial databricks"  
[YouTube Link](https://youtube.com/watch?v=QhaELILKk38)  2021-07-13T10:42Z 38K followers, 80.8K engagements


"33. Databricks Spark Pyspark UDF #SparkUDF #DatabricksUDF #UDF#UserDefinedFunction #PysparkUDF #Databricks #DatabricksTutorial #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure databricks notebook tutorial databricks delta lake databricks azure tutorial Databricks Tutorial for beginners azure Databricks tutorial databricks tutorial databricks community edition databricks community edition cluster creation databricks community edition tutorial"  
[YouTube Link](https://youtube.com/watch?v=V5HEbR782nI)  2021-08-17T12:30Z 36.8K followers, 20.3K engagements


"37. Databricks Pyspark: Dataframe Checkpoint Azure Databricks Learning: ================== What is dataframe Checkpointing in Spark/Databricks This video explains more about dataframe checkponting in databricks development. #DatabricksCheckpoint #DataframeCheckpoint #SparkCheckpoint #SparkCache#DatabricksCache #PysparkCheckpoint #SparkPersist #DatabricksPersist #DataframePersist#DatabricksRealtime #SparkRealTime #DatabricksInterviewQuestion #DatabricksInterview #SparkInterviewQuestion #SparkInterview #PysparkInterviewQuestion #PysparkInterview #BigdataInterviewQuestion"  
[YouTube Link](https://youtube.com/watch?v=hgzSL8bnJFQ)  2022-02-15T12:30Z 36.8K followers, 25.6K engagements


"23. Databricks Spark Cache vs Persist Interview Question Performance Tuning #Cache #Persist #DatabricksOptimization #SparkOptimization #CachevsPersist #DatabricksInterviewQuestions #SparkInterviewQuestions #DatabricksInterview #DatabricksPerformance #Databricks #DatabricksTutorial #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure databricks notebook tutorial databricks delta lake databricks azure tutorial Databricks Tutorial for beginners azure"  
[YouTube Link](https://youtube.com/watch?v=p6_0qdd6X08)  2021-07-19T13:17Z 37.9K followers, 41.9K engagements


"74. Databricks Pyspark Interview Question: Sort-Merge Join (SMJ) Azure Databricks Learning: Sort Merge Join ========================================== What is sort-merge join in Spark Sort-merge join is one of the internal joining mechanism used by spark to join multiple dataframes. It is important to understand th internal working mechanism to understand the performance of spark program. This is also one of the widely asked interview question #SortMergeJoin #SparkSortMerge #SparkInternalJoin #BroadcastJoin #ShuffleHashJoin#DatabricksSortMergeJoin #DatabricksRealtime #SparkRealTime"  
[YouTube Link](https://youtube.com/watch?v=DFtvA5O58X4)  2022-07-07T12:30Z 38.3K followers, 25.1K engagements


"61. Databricks Pyspark Delta Lake : Slowly Changing Dimension (SCD Type2) Azure Databricks Learning: ================== How to handle Slowly Changing Dimension Type2 (SCD Type2) requirement in Databricks using Pyspark This video covers end to end development steps of SCD Type [--] using Pyspark in Databricks environment #DatabricksSCDType2 #SCDType2 #SparkSCDType2#PySparkSCDType2#SlowlyChangingDimenson2 #DatabricksSlowlyChangingDimension2 #DatabricksPerformanceOptimization #DatabricksScenarioBasedInterviewQuestion #SparkScenarioBasedInterviewQuestion #DatabricksReadCsvInterviewQuestion"  
[YouTube Link](https://youtube.com/watch?v=GhBlup-8JbE)  2022-05-15T12:30Z 38.4K followers, 69K engagements


"121. Databricks Pyspark AutoLoader: Incremental Data Load Azure Databricks Learning: Databricks and Pyspark: AutoLoader: Incremental Data Load ===================================================================================== AutoLoader in Databricks is a crucial feature that streamlines the process of ingesting and processing large volumes of data efficiently. This automated data loading mechanism is instrumental for real-time or near-real-time data pipelines allowing organizations to keep their data lakes up-to-date with minimal manual intervention. By automatically detecting and loading"  
[YouTube Link](https://youtube.com/watch?v=GjV2m8b9fNY)  2023-11-14T12:30Z 38.4K followers, 37K engagements


"67. Databricks Pypark Delta: Schema Evolution - MergeSchema Azure Databricks Learning: Delta Lake - Schema Evolution: Merge Schema ======================================================================= How to handle Schema mismatch scenario in delta lake development Schema Evolution is one of the common scenarioin today's moden big data world. It is important to put a mechanism to handle schema mismatch to avoid pipeline failure This video gives complete information about MergeSchema in databricks #DatabricksSchemaEvolution #SchemaEvolution #MergeSchema #SchemaMismatch#DeltaSchemaEvolution"  
[YouTube Link](https://youtube.com/watch?v=NOYL0yRoUeo)  2022-06-30T12:30Z 38.4K followers, 25.5K engagements


"84. Databricks Pyspark Azure Data Factory + Azure Databricks: Execute Notebook Via ADF Azure Databricks Learning: Execute Azure Databricks Notebook through Azure Data Factory ================================================================================ How can we execute the databricks notebook from Azure data factory Azure data factory is known for its scheduling features so it is chosen as a scheduler for most of the projects. To execute the notebook from ADF Azure Data Factory provides Notebook activity through which notebook can be invoked. To get through understanding of this concept"  
[YouTube Link](https://youtube.com/watch?v=fLZ3b9uiPEI)  2022-12-01T12:30Z 38.4K followers, 20.1K engagements


"59. Databricks Pyspark:Slowly Changing DimensionSCD Type1 Merge using Pyspark and Spark SQL #DatabricksMerge#DatabricksUpsert #SparkMerge#SparkUpsert#PysparkMerge#PysparkUpsert#SparkSqlMerge#SparksqlUpsert#SlowlyChangingDimension #SCDType #SCDType1 #DatabricksWhenMatched #DatabricksWhenNotMatched #Deltalake #Deltatable #DeltaMerge #DeltaUpsert #DatabricksTutorial #DatabricksMergeStatement #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure"  
[YouTube Link](https://youtube.com/watch?v=i5oM2bUyH0o)  2022-01-04T12:30Z 38.4K followers, 53.3K engagements


"51. Databricks Pyspark Delta Lake: Introduction to Delta Lake Azure Databricks Learning: Delta Lake ==================================== What is Delta Lake This video covers differences between data warehouse Data lake and Delta lake. Convers the introduction to delta lake in databricks #DeltalakeIntro #IntroductionToDeltaLake #Deltalake #DeltaTable #DatabricksDelta #DeltaTableCreate #DatawarehouseVsDataLakevsDeltaLake #PysparkDeltaLake #DeltalakevsDatalake #SQLDeltaTable #DataframeDeltaTable#DeltaFormat #DatabricksRealtime #SparkRealTime #DatabricksInterviewQuestion #DatabricksInterview"  
[YouTube Link](https://youtube.com/watch?v=t6i6fQilAm8)  2022-04-16T12:30Z 38.3K followers, 86.8K engagements


"92. Databricks Pyspark Interview Question Performance Optimization: Select vs WithColumn Azure Databricks Learning: Interview Question Performance Optimization: Select vs WithColumn ================================================================================ What is the difference between pyspark functions select and withcolumn Select and withcolumn both are used to add new columns to existing dataframe. But select outperforms withcolumn. The reason behind this difference is explained in this video. To get through understanding of this concept please watch this video"  
[YouTube Link](https://youtube.com/watch?v=-Q7xNTPcEFA)  2022-12-14T12:30Z 38.4K followers, 12.3K engagements


"36. Databricks: Autoscaling Optimized Autoscaling Azure Databricks Learning: ================== Databricks Interview Question: What is Autoscaling what are the types of Autoscaling What i optimized Autoscaling What is the importance of Autoscaling To get answer and more details to above questions please watch this video. #DatabricksAutoscaling #DatabricksOptimizedAutoscaling #DatabricksStandardAutoscaling #DatabricksPerformanceOptimization #DatabricksCostSaving #DatabricksScenarioBasedInterviewQuestion #SparkScenarioBasedInterviewQuestion #DatabricksReadCsvInterviewQuestion #SparkJobs"  
[YouTube Link](https://youtube.com/watch?v=05O1f4zCnxg)  2023-10-23T19:50Z 14.2K followers, [----] engagements


"118. Databricks PySpark SQL Coding Interview: Employees Earning More than Managers Azure Databricks Learning: Coding Interview Exercise: Pyspark and Spark SQL ================================================================================= Coding exercises are very common in most of the Bigdata interviews. It is important to develop coding skills before appearing for Spark/Databricks interviews. In this video I have explained a coding scenario to find the employees who are earning more than their managers. This is Leet Code SQL Exercise number [----]. This is also one of common coding exercise"  
[YouTube Link](https://youtube.com/watch?v=1fVquYHIHig)  2023-10-03T12:29Z 38.4K followers, [----] engagements


"62. Databricks Pyspark Delta Lake: Time Travel Azure Databricks Learning: Delta Lake Time Travel ================================================== What is Time Travel in delta table and how to perform time travel Time Travel is one of key feature provided by Databricks for Delta lake development using which we can travel back and forth of snapshots of delta table. There are various appoached to perform time travel. Have covered around [--] different approaches in this video #DeltaTimeTravel #DeltaLakeVersion #VersionAsOf #TimestampAsOf #DatabriksTimeTravel #DeltalakeIntro"  
[YouTube Link](https://youtube.com/watch?v=3av7ctZ1uoo)  2023-10-21T11:40Z 14.2K followers, [----] engagements


"06. Databricks Pyspark Spark Reader: Read CSV File #ReadCSV #DatabricksCSVFile #DataframeCSV #Databricks #DatabricksTutorial #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure databricks notebook tutorial databricks delta lake databricks azure tutorial Databricks Tutorial for beginners azure Databricks tutorial databricks tutorial databricks community edition databricks community edition cluster creation databricks community edition tutorial"  
[YouTube Link](https://youtube.com/watch?v=7mfKuo_Ng_Q)  2023-10-26T13:44Z 14.2K followers, 35K engagements


"66. Databricks Pyspark Delta: Z-Order Command Azure Databricks Learning: Delta Lake - Z-Order Command ======================================================== What is Z-order Command in delta table and how to apply in delta lake development Z-order one of the performance optimization techinique used in delta lake. It is used along with optimize command and used to compact small files into optimal size and at the same time relevant data is co-located to improve the performance. This video gives complete understanding of Z-order command #DeltaZorder #DatabricksZorder #PerformanceOptimization"  
[YouTube Link](https://youtube.com/watch?v=89cInDvqXCY)  2022-06-29T12:30Z 27.3K followers, 23.9K engagements


"101. Databricks Pyspark Core/Architecture: Spark/Databricks Interview Question Series - I Azure Databricks Learning: Spark/Databricks Interview Questions on Core/Architecture Concepts -I ================================================================================= Are you learning Spark/Databricks to get a job Are you preparing for Interview for the role of Spark/Databricks data engineer Follow this video to get list of questions on spark/Databricks core concepts along with directions to give answer in the interview #SparkInterviewQuestions"  
[YouTube Link](https://youtube.com/watch?v=AUvnhHeHriA)  2023-05-18T15:32Z 38.4K followers, 12.9K engagements


"119. Databricks Pyspark Spark SQL: Except Columns in Select Clause Azure Databricks Learning: Pyspark and Spark SQL: Except Columns in Select Clause ================================================================================= Except function provided by Databricks in Spark SQL is powerful feature while performing data analytics of dataset with 1000s of columns. It is life saver feature for developers for data engineering and data Analytics projects To get more understanding watch this video https://youtu.be/Aj0kTlD9IgI #ExceptColumns"  
[YouTube Link](https://youtube.com/watch?v=Aj0kTlD9IgI)  2023-10-06T12:30Z 38.4K followers, [----] engagements


"63. Databricks Pyspark Delta Lake: Restore Command Azure Databricks Learning: Delta Lake - Restore Command ======================================================== What is Restore Command in delta table and how to apply in delta lake development Restore command is one of key feature provided by Databricks for Delta lake development using which we can restore the delta table permanently to previous state/version/timestamp. There are various approaches to apply restore command. Have talked about [--] different approaches in this video #DeltaRestore#DatabricksRestoreCommand #DeltaLakeVersion"  
[YouTube Link](https://youtube.com/watch?v=CHfP2UxZn1g)  2022-05-22T12:30Z 27K followers, [----] engagements


"64. Databricks Pyspark Delta Lake: Optimize Command - File Compaction Azure Databricks Learning: Delta Lake - Optimize Command ======================================================== What is Optimize Command in delta table and how to apply in delta lake development Optimize is one of the performance optimization techinique used in delta lake. It compacts the smaller size files into optimal size. This video talks more about optimize command #DeltaOptimize #DatabricksOptimize #PerformanceOptimization #Optimize #DeltaCompactFiles #DeltaSmallFileIssue #DeltalakePerformance"  
[YouTube Link](https://youtube.com/watch?v=F9tc8EgIn3c)  2022-06-01T12:30Z 27K followers, 19.4K engagements


"102. Databricks Pyspark Performance Optimization: Spark/Databricks Interview Question Series - II Azure Databricks Learning: Performance Optimization: Spark/Databricks Interview Question Series - II ================================================================================= Are you learning Spark/Databricks to become Bigdata Engineer Are you preparing for an Interview for the role of Spark/Databricks data engineer Performance optimization is one of the constant topic in all interview calls. Follow this video to get list of questions on performance optimization concepts along with"  
[YouTube Link](https://youtube.com/watch?v=FTDecDchRkw)  2023-05-30T12:30Z 38.4K followers, 14.2K engagements


"65. Databricks Pyspark Delta Lake: Vacuum Command Azure Databricks Learning: Delta Lake - Vacuum Command ======================================================== What is Vacuum Command in delta table and how to apply in delta lake development Vacuum is one of the performance optimization techinique used in delta lake. It removes obsolete files from delta table folder This video talks more about vacuum command #DeltaVacuum #DatabricksVacuum #PerformanceOptimization #Vacuum #DeltaCompactFiles #DeltaSmallFileIssue #DeltalakePerformance #DeltaPerformanceImprovement #DeltalakeIntro"  
[YouTube Link](https://youtube.com/watch?v=G_RzisFeA5U)  2022-06-24T12:30Z 27K followers, 17.7K engagements


"19. Databricks & Pyspark: Real Time ETL Pipeline Azure SQL to ADLS Azure Databricks Learning: ========================== How to create ETL Pipeline to load data from Azure SQL to Azure Data Lake Storage This video covers end to end process to create end to end ETL pipeline to load data from Azure SQL to ADLS. This demo exercise covers these three areas [--]. Extract data from Azure SQL tables [--]. Transform the data with business rules [--]. Load the data to Azure Data Lake Storage"  
[YouTube Link](https://youtube.com/watch?v=Ia6fDlhlKXQ)  2022-01-17T12:30Z 27.2K followers, 48.2K engagements


"83. Databricks Pyspark Databricks Workflows: Job Scheduling Azure Databricks Learning: Databricks Workflows: Job Scheduling ======================================================== How to create jobs schedule them in Databricks development Development of ETL pipelines and scheduling them go hand in hand in any data engineering projects. Databricks provides this feature in the form of workflows. Workflows creates jobs in the form of collection of task and gives the provision of schedule. To get through understanding of this concept please watch this video #DatabricksWorkflows#DatabricksJobs"  
[YouTube Link](https://youtube.com/watch?v=ODqba9BAPvs)  2022-11-30T12:30Z 35.7K followers, 36.9K engagements


"05. Databricks Pyspark: Cluster Deployment #DatabricksCluster #Clusterdeployment #Sparkcluster #Databricks #DatabricksTutorial #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure databricks notebook tutorial databricks delta lake databricks azure tutorial Databricks Tutorial for beginners azure Databricks tutorial databricks tutorial databricks community edition databricks community edition cluster creation databricks community edition tutorial"  
[YouTube Link](https://youtube.com/watch?v=OzgOJfztsFY)  2021-07-10T16:55Z 35.6K followers, 54.2K engagements


"54. Databricks Delta Lake Pyspark: Create Delta Table Using Various Methods Azure Databricks Learning: Delta Lake ======================================================= How to create delta table in databricks development Delta table can be created using various methods in databricks. In this tutorial the most commonly used [--] approaches are covered [--]. Using Pyspark without databricks [--]. Using Spark SQL [--]. Using dataframe with data #Deltalake #DeltaTable #DatabricksDelta #DeltaTableCreate #SparkSQL #PysparkDeltaLake #PysparkDeltaTable #SQLDeltaTable #DataframeDeltaTable#DeltaFormat"  
[YouTube Link](https://youtube.com/watch?v=RTIcUB_oi4E)  2022-04-14T12:30Z 35.8K followers, 55.4K engagements


"26. Databricks Spark Adaptive Query Execution Interview Question Performance Tuning #AdaptiveQueryExecution #DatabricksOptimization #SparkOptimization #AQE #DatabricksInterviewQuestions #SparkInterviewQuestions #DatabricksInterview #DatabricksPerformance #Databricks #DatabricksTutorial #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure databricks notebook tutorial databricks delta lake databricks azure tutorial Databricks Tutorial for beginners"  
[YouTube Link](https://youtube.com/watch?v=SK0Rit3GmKE)  2021-07-26T17:30Z 32.2K followers, 32.1K engagements


"08. Databricks Pyspark: Add Rename and Drop Columns #Databricks #DatabricksTutorial #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure databricks notebook tutorial databricks delta lake databricks azure tutorial Databricks Tutorial for beginners azure Databricks tutorial databricks tutorial databricks community edition databricks community edition cluster creation databricks community edition tutorial databricks community edition pyspark databricks"  
[YouTube Link](https://youtube.com/watch?v=Tc_Dk80ukYo)  2023-10-24T19:08Z 14.2K followers, 14.6K engagements


"120. Databricks Pyspark SQL Coding Interview: Employees Earning More Than Department Avg Salary Azure Databricks Learning: Coding Interview Exercise: Pyspark and Spark SQL ================================================================================= Coding exercises are very common in most of the Bigdata interviews. It is important to develop coding skills before appearing for Spark/Databricks interviews. In this video I have explained a coding scenario to find the employees who are earning more than their department average salary. This is also one of common coding exercise asked in"  
[YouTube Link](https://youtube.com/watch?v=Tjv881etqgY)  2023-10-10T12:30Z 38.4K followers, [----] engagements


"103. Databricks Pyspark Delta Lake: Spark/Databricks Interview Question Series - III Azure Databricks Learning: Delta Lake: Spark/Databricks Interview Question Series - III ================================================================================= Are you learning Spark/Databricks to become Bigdata Engineer Are you preparing for an Interview for the role of Spark/Databricks data engineer Delta Lake is one of the modern Lakehouse concept which is used on most of bigdata projects. This is also one of the major topic to crack interviews. Follow this video to get list of questions on Delta"  
[YouTube Link](https://youtube.com/watch?v=VK9ZA688GuM)  2023-06-13T12:30Z 38.4K followers, 11.8K engagements


"49. Databricks & Spark: Interview Question(Scenario Based) - How many spark jobs get created Azure Databricks Learning: ================== Scenario Based Interview Question: How many spark jobs get created while reading CSV file with different options This video covers more details about spark csv reading scenario. This interview question is based on real time scenario. #DatabricksScenarioBasedInterviewQuestion #SparkScenarioBasedInterviewQuestion #DatabricksReadCsvInterviewQuestion #SparkJobs #NumberofSparkJobs #DatabricksSparkJobs#DatabricksRealtime #SparkRealTime"  
[YouTube Link](https://youtube.com/watch?v=VLi9WS8SJFY)  2023-10-23T06:24Z 14.2K followers, [----] engagements


"31. Databricks Pyspark: Handling Null - Part1 #NullHandle #PysparkNull #DatabricksNull #DataframeNull #RDDNull #Databricks #DatabricksTutorial #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure databricks notebook tutorial databricks delta lake databricks azure tutorial Databricks Tutorial for beginners azure Databricks tutorial databricks tutorial databricks community edition databricks community edition cluster creation databricks community"  
[YouTube Link](https://youtube.com/watch?v=XPpMmKofgx4)  2021-07-11T13:26Z 34.4K followers, 19.3K engagements


"52. Databricks Pyspark Delta Lake Architecture: Internal Working Mechanism Azure Databricks Learning: Delta Lake Architecture ================================================== What is Internal working mechanism of Delta Lake This video covers delta lake architecture with deep knowledge of internal working mechanisms. It is important for every databricks developer to understand delta lake internals #DeltaLakeArchitecture #DeltaLakeInternal #DeltalakeInternalMechanism #DeltaInternalWorkingMechanism #DeltaTransactionLog #DeltaCheckpointFile #DeltaCRC #DeltaJsonTransactionFile #DeltaLogFile"  
[YouTube Link](https://youtube.com/watch?v=YmqkMZ4MxJg)  2022-04-18T15:29Z 27.2K followers, 46K engagements


"133: Databricks Certification: Data Engineer Associate - PII Comment Welcome to our YouTube series designed to help you ace the Databricks Certified Data Engineer Associate certification This series provides comprehensive coverage of all exam objectives practical demos and expert tips to ensure you gain the skills and confidence needed to succeed. Whether you're a beginner or looking to sharpen your Databricks expertise this series is your go-to resource. Join us on this journey to elevate your data engineering career 🚀"  
[YouTube Link](https://youtube.com/watch?v=bXqErQc3oEY)  2025-01-03T12:30Z 38.4K followers, [----] engagements


"114. Databricks Pyspark Performance Optimization: Re-order Columns in Delta Table Azure Databricks Learning: Delta Lake: How to re-order columns of a delta table ================================================================================= Re-ordering tables columns is one of the most common requirement in database and data warehousing concepts. It is also improving performance in Databricks delta lake. To know more about it watch this video https://youtu.be/cnWmN8T6E9I #Deltalake #DataSkipping #DeltaSkipping #ReorderDeltaColumns # RepositionDeltaColumns"  
[YouTube Link](https://youtube.com/watch?v=cnWmN8T6E9I)  2023-09-05T12:30Z 38.4K followers, [----] engagements


"89. Databricks Pyspark Notebook Scheduling through Event Based Trigger using Azure Data Factory Azure Databricks Learning: Notebook Scheduling through Event Based Trigger using ADF ================================================================================ How to schedule Databricks Notebook through event based trigger using Azure Data Factory ADF plays pivotal role in scheduling various ETL pipelines through ADF activities. Azure databricks notebook can be scheduled through notebook activity in Azure data factory. Event based trigger creates job scheduling as soon as file arrives. To"  
[YouTube Link](https://youtube.com/watch?v=dP-ZXgxx5TY)  2022-12-06T12:30Z 38.4K followers, 13.3K engagements


"87. Databricks Pyspark Real Time Project: ETL Pipeline Integrating ADF ASQL ADLS Key Vault Azure Databricks Learning: Real Time Project:ETL Pipeline Integrating Databricks ADF ASQL ADLS and Key Vault ================================================================================ How to develop ETL Pipeline Integrating Databricks ADF ASQL ADLS and Key Vault This tutorial explains the real time project scenario of ETL pipeline development of integrating Azure Databricks with ADF ASQL ADLS and Key Vault To get through understanding of this concept please watch this video"  
[YouTube Link](https://youtube.com/watch?v=dxxXWe4gNTo)  2022-12-04T12:30Z 38.4K followers, 23K engagements


"29. Azure Synapse Analytics ADW Architecture MPP Part [--] #AzureSynapseAnalytics #AzureDWH #AzureDWHArchitecture #AzureMPP #MassivelyParallelProcessing #MPP #AzureDataWarehouse #AzureArchitecture"  
[YouTube Link](https://youtube.com/watch?v=hzu-iZHMHOM)  2021-07-20T14:00Z 33.1K followers, [----] engagements


"85. Databricks Pyspark Notebook Activity in Azure Data Factory with Input Parameter Azure Databricks Learning: Execute Azure Databricks Notebook through Azure Data Factory with Input Paramters ================================================================================ How can we execute the Databricks notebook from Azure data factory with input parameters This tutorial explains the process of running databricks notebook through notebook activity in Azure Data Factory with Input parameters To get through understanding of this concept please watch this video #DatabricksWidgets"  
[YouTube Link](https://youtube.com/watch?v=ldVgPhjaB7w)  2022-12-02T12:30Z 38.4K followers, 16.3K engagements


"97. Databricks Pyspark Data Security: Enforcing Column Level Encryption Azure Databricks Learning: Data Security: Enforcing Column Level Encryption =================================================================== How to implement data security features in Databricks Development Data security is of utmost importance in Databricks for several reasons such as Compliance with regulations Maintaining trust with customers Preventing data breaches etc. In this video I have explained how to enforce column-level encryption in databricks development To get through understanding of this concept"  
[YouTube Link](https://youtube.com/watch?v=nu5_dOKAJcg)  2023-03-23T13:05Z 38.4K followers, 14.3K engagements


"122. Databricks Pyspark Delta Live Table: Introduction Delta Live Table Tutorial: [--]. Delta Lake Internal Architecture - https://youtu.be/YmqkMZ4MxJgsi=GbX3Fi1SH4sb_elw [--]. Auto Loader - https://youtu.be/GjV2m8b9fNYsi=gY9K3MISDYkRlImA [--]. DLT Introduction - https://youtu.be/ryOe64wwLuwsi=JS-izYpggbm1H1Wp Azure Databricks Learning: Databricks and Pyspark: Delta Live Table: Introduction ===================================================================================== Delta Live Tables in Databricks is a groundbreaking feature that takes data processing to the next level. It brings the power of"  
[YouTube Link](https://youtube.com/watch?v=ryOe64wwLuw)  2023-11-20T12:30Z 38.4K followers, 32.7K engagements


"94. Databricks Pyspark Interview Question Schema Definition: Struct Type vs Map Type Azure Databricks Learning: Interview Question Schema Definition: Struct Type vs Map Type ================================================================================ What is the difference between pyspark methods StructType and Maptype StructType and Maptype both are used to define structure of a nested field dataframe. But both are used for different use cases. I have explained the difference in this video To get through understanding of this concept please watch this video"  
[YouTube Link](https://youtube.com/watch?v=wI-nqFPW580)  2022-12-20T12:31Z 38.4K followers, [----] engagements


"21. Databricks Spark Streaming #DatabricksStreaming #SparkStreaming #Streaming #Databricks #DatabricksTutorial #AzureDatabricks #Databricks #Pyspark #Spark #AzureDatabricks #AzureADF #Databricks #LearnPyspark #LearnDataBRicks #DataBricksTutorial databricks spark tutorial databricks tutorial databricks azure databricks notebook tutorial databricks delta lake databricks azure tutorial Databricks Tutorial for beginners azure Databricks tutorial databricks tutorial databricks community edition databricks community edition cluster creation databricks community edition tutorial databricks community"  
[YouTube Link](https://youtube.com/watch?v=zx5H82fmUPU)  2021-07-14T14:46Z 35.7K followers, 47.1K engagements


"133: Databricks Certification: Data Engineer Associate - PII Comment Welcome to our YouTube series designed to help you ace the Databricks Certified Data Engineer Associate certification This series provides comprehensive coverage of all exam objectives practical demos and expert tips to ensure you gain the skills and confidence needed to succeed. Whether you're a beginner or looking to sharpen your Databricks expertise this series is your go-to resource. Join us on this journey to elevate your data engineering career 🚀"  
[YouTube Link](https://youtube.com/watch?v=bXqErQc3oEY)  2025-01-03T12:30Z 38.4K followers, [----] engagements


"132: DataBricks Learning: System Variable _SQLDF Dive deep into Databricks _sqldf function This video explains how to leverage the power of SQL within your Python code for data manipulation and analysis. Learn how to easily query and transform your data using familiar SQL syntax all within the Databricks environment. Whether you're a beginner or an experienced user this video will provide valuable insights into this powerful feature"  
[YouTube Link](https://youtube.com/watch?v=FHgNPHBtK8o)  2025-01-02T12:30Z 38.4K followers, [----] engagements


"131. Databricks Pyspark Built-in Function: ZIP_WITH [---]. Databricks Pyspark Built-in Function: ZIP_WITH ============================================ 🚀 New YouTube Video Alert 🚀 I just released a new video on YouTube where I dive into the powerful zip_with function in PySpark 📊🔧 I am excited to announce the release of my latest YouTube video where I delve into the powerful Change Data Feed (CDF) feature in Databricks. 📊✨ In this video you'll learn: The basics of the zip_with function. Practical examples of using zip_with for element-wise operations. How to apply custom binary functions to"  
[YouTube Link](https://youtube.com/watch?v=8LVmUpFLMzA)  2024-07-22T12:30Z 38.4K followers, [----] engagements


"130. Databricks Pyspark Delta Lake: Change Data Feed [---]. Databricks Pyspark Delta Lake: Change Data Feed ======================================================== 🚀 New YouTube Video Alert: Exploring Change Data Feed in Databricks 🚀 I am excited to announce the release of my latest YouTube video where I delve into the powerful Change Data Feed (CDF) feature in Databricks. 📊✨ In this video you'll learn: 🔹 What Change Data Feed is and how it works 🔹 How to enable and use CDF in your Databricks environment 🔹 Practical examples showcasing real-time data processing and analytics Whether"  
[YouTube Link](https://youtube.com/watch?v=asm_oT6fKf0)  2024-07-15T12:30Z 38.4K followers, 14.4K engagements


"129. Databricks Pyspark Delta Lake: Deletion Vectors [---]. Databricks Pyspark Delta Lake: Deletion Vectors ======================================================== Delta Lake Internal Architecture: https://youtu.be/YmqkMZ4MxJgsi=EEgkoZZKJ7F4QsaH Optimize Command : https://youtu.be/F9tc8EgIn3csi=9KknJFJeHJunYJ_h Vacuum Command : https://youtu.be/G_RzisFeA5Usi=FDNusdn2U4vjIlup 🚀 Excited to announce my latest YouTube video on the new Databricks Deletion Vectors feature 🎥 In this video I dive deep into how Databricks Deletion Vectors enable efficient and scalable data deletion without physically"  
[YouTube Link](https://youtube.com/watch?v=Q-IY1CxK_r4)  2024-07-08T12:30Z 38.4K followers, [----] engagements


"128. Databricks Pyspark Built-In Function: TRANSFORM [---]. Databricks Pyspark Built-In Function: TRANSFORM The transform function in PySpark is a versatile and powerful feature that plays a crucial role in data engineering and data science use cases. In this tutorial video learn how to develop concise and more readable solution in Databricks development. https://youtu.be/eNUYxJBMrh8 #Databricks #TRANSFORM #PysparkBuilt-InFunction #DataEngineering #DataScience #Tutorial #LinkedInLearning #TechTutorial #DataAnalytics #DataManagement #YouTubeTutorial #Databricks #AutoLoader #DataIngestion"  
[YouTube Link](https://youtube.com/watch?v=eNUYxJBMrh8)  2024-07-03T12:30Z 38.4K followers, [----] engagements


"127. Databricks Pyspark SQL Coding Interview:LeetCode-1045: Customers Who Bought All Products Azure Databricks Learning: Coding Interview Exercise: Pyspark and Spark SQL ================================================================================= Coding exercises are very common in most Big Data interviews. It is important to develop coding skills before appearing for Spark/Databricks interviews. In this video I have explained a coding scenario to find the customers who bought all the products available. This is also one of the common coding exercises asked in MAANG/FAANG/GAMAM companies"  
[YouTube Link](https://youtube.com/watch?v=zO1QljAJ834)  2024-05-28T12:30Z 38.4K followers, [----] engagements


"126. Databricks Pyspark Downloading Files from Databricks DBFS Location Quick Guide: Downloading Files from Databricks DBFS Location ============================================================ In this short tutorial video learn how to effortlessly download files from a Databricks DBFS (Databricks File System) location. Whether you're a data engineer data scientist or analyst working with Databricks accessing and retrieving files from DBFS is a fundamental skill. * Accessing DBFS: Learn how to navigate to the DBFS location containing the files you want to download within the Databricks"  
[YouTube Link](https://youtube.com/watch?v=-XH23MI9L7w)  2024-04-02T12:30Z 38.4K followers, [----] engagements


"125. Databricks Pyspark Delta Live Table: Data Quality Check - Expect Azure Databricks Learning: Databricks and Pyspark: Delta Live Table: Data Quality Check - Expect ================================================================================================ 🚀 Excited to share my latest YouTube video discussing the powerful data quality checks feature of "expect" in Delta Live Tables on Databricks In today's data-driven world ensuring data accuracy and reliability is paramount. With "expect" we can effortlessly define and enforce data quality constraints streamlining our data pipelines"  
[YouTube Link](https://youtube.com/watch?v=OUtmiA56Rfk)  2024-03-01T12:30Z 38.4K followers, [----] engagements


"124. Databricks Pyspark Delta Live Table: Datasets - Tables and Views Delta Live Table Tutorial: [--]. Delta Lake Internal Architecture - https://youtu.be/YmqkMZ4MxJgsi=GbX3Fi1SH4sb_elw [--]. Auto Loader - https://youtu.be/GjV2m8b9fNYsi=gY9K3MISDYkRlImA [--]. DLT Introduction - https://youtu.be/ryOe64wwLuwsi=JS-izYpggbm1H1Wp [--]. DLT Declarative vs Procedural - https://youtu.be/-ia78A2QMN0si=MgkO7zfwYRjK6843 [--]. DLT Datasets - https://youtu.be/4QatH7WBSeksi=P2Hy01ozp8SeDLxY Azure Databricks Learning: Databricks and Pyspark: Delta Live Table: Datasets - Tables and Views"  
[YouTube Link](https://youtube.com/watch?v=4QatH7WBSek)  2023-11-30T12:30Z 38.4K followers, 14.5K engagements


"123. Databricks Pyspark Delta Live Table: Declarative VS Procedural Delta Live Table Tutorial: [--]. Delta Lake Internal Architecture - https://youtu.be/YmqkMZ4MxJgsi=GbX3Fi1SH4sb_elw [--]. Auto Loader - https://youtu.be/GjV2m8b9fNYsi=gY9K3MISDYkRlImA [--]. DLT Introduction - https://youtu.be/ryOe64wwLuwsi=JS-izYpggbm1H1Wp [--]. DLT Declarative vs Procedural - https://youtu.be/-ia78A2QMN0si=MgkO7zfwYRjK6843 Azure Databricks Learning: Databricks and Pyspark: Delta Live Table: Introduction ===================================================================================== Understanding the declarative"  
[YouTube Link](https://youtube.com/watch?v=-ia78A2QMN0)  2023-11-27T12:30Z 38.4K followers, 13.4K engagements


"122. Databricks Pyspark Delta Live Table: Introduction Delta Live Table Tutorial: [--]. Delta Lake Internal Architecture - https://youtu.be/YmqkMZ4MxJgsi=GbX3Fi1SH4sb_elw [--]. Auto Loader - https://youtu.be/GjV2m8b9fNYsi=gY9K3MISDYkRlImA [--]. DLT Introduction - https://youtu.be/ryOe64wwLuwsi=JS-izYpggbm1H1Wp Azure Databricks Learning: Databricks and Pyspark: Delta Live Table: Introduction ===================================================================================== Delta Live Tables in Databricks is a groundbreaking feature that takes data processing to the next level. It brings the power of"  
[YouTube Link](https://youtube.com/watch?v=ryOe64wwLuw)  2023-11-20T12:30Z 38.4K followers, 32.7K engagements


"121. Databricks Pyspark AutoLoader: Incremental Data Load Azure Databricks Learning: Databricks and Pyspark: AutoLoader: Incremental Data Load ===================================================================================== AutoLoader in Databricks is a crucial feature that streamlines the process of ingesting and processing large volumes of data efficiently. This automated data loading mechanism is instrumental for real-time or near-real-time data pipelines allowing organizations to keep their data lakes up-to-date with minimal manual intervention. By automatically detecting and loading"  
[YouTube Link](https://youtube.com/watch?v=GjV2m8b9fNY)  2023-11-14T12:30Z 38.4K followers, 37K engagements


"120. Databricks Pyspark SQL Coding Interview: Employees Earning More Than Department Avg Salary Azure Databricks Learning: Coding Interview Exercise: Pyspark and Spark SQL ================================================================================= Coding exercises are very common in most of the Bigdata interviews. It is important to develop coding skills before appearing for Spark/Databricks interviews. In this video I have explained a coding scenario to find the employees who are earning more than their department average salary. This is also one of common coding exercise asked in"  
[YouTube Link](https://youtube.com/watch?v=Tjv881etqgY)  2023-10-10T12:30Z 38.4K followers, [----] engagements


"119. Databricks Pyspark Spark SQL: Except Columns in Select Clause Azure Databricks Learning: Pyspark and Spark SQL: Except Columns in Select Clause ================================================================================= Except function provided by Databricks in Spark SQL is powerful feature while performing data analytics of dataset with 1000s of columns. It is life saver feature for developers for data engineering and data Analytics projects To get more understanding watch this video https://youtu.be/Aj0kTlD9IgI #ExceptColumns"  
[YouTube Link](https://youtube.com/watch?v=Aj0kTlD9IgI)  2023-10-06T12:30Z 38.4K followers, [----] engagements


"118. Databricks PySpark SQL Coding Interview: Employees Earning More than Managers Azure Databricks Learning: Coding Interview Exercise: Pyspark and Spark SQL ================================================================================= Coding exercises are very common in most of the Bigdata interviews. It is important to develop coding skills before appearing for Spark/Databricks interviews. In this video I have explained a coding scenario to find the employees who are earning more than their managers. This is Leet Code SQL Exercise number [----]. This is also one of common coding exercise"  
[YouTube Link](https://youtube.com/watch?v=1fVquYHIHig)  2023-10-03T12:29Z 38.4K followers, [----] engagements


"117. Databricks Pyspark SQL Coding Interview: Total Grand Slam Titles Winner Azure Databricks Learning: Coding Interview Exercise: Pyspark and Spark SQL ================================================================================= Coding exercises are very common in most of the Bigdata interviews. It is important to develop coding skills before appearing for Spark/Databricks interviews. In this video I have explained a coding scenario to find the total number grand slam titles won by each player. This is Leet Code SQL Exercise number [----]. This is also one of common coding exercise asked"  
[YouTube Link](https://youtube.com/watch?v=uKLEgMmqees)  2023-09-19T12:30Z 38.4K followers, [----] engagements


"116. Databricks Pyspark Query Dataframe Using Spark SQL Azure Databricks Learning: Query Dataframe Using Spark SQL ========================================================== SQL is one of the most convenient language for most of the data engineers across globe. Though Spark provides the feature of exploring Dataframe using SQL only after converting it to table or view Spark has introduced new feature in [---] onwards through which no need of converting dataframe into table or view. To understand this feature better watch this video https://youtu.be/pjjIK82_Vsc #DataframeinSparkSQL #SparkSQL"  
[YouTube Link](https://youtube.com/watch?v=pjjIK82_Vsc)  2023-09-18T12:30Z 38.4K followers, [----] engagements


"115. Databricks Pyspark SQL Coding Interview: Number of Calls and Total Duration Azure Databricks Learning: LeetCode Coding Interview Exercise: Pyspark and Spark SQL ================================================================================= Coding exercises are very common in most of the Bigdata interviews. It is important to develop coding skills before appearing for Spark/Databricks interviews. In this video I have explained a coding scenario to calculate the number of calls and total call duration between [--] persons. This is Leet Code SQL Exercise number [----]. This is also one of"  
[YouTube Link](https://youtube.com/watch?v=M6E3PiNKIkM)  2023-09-08T12:30Z 38.4K followers, [----] engagements


"114. Databricks Pyspark Performance Optimization: Re-order Columns in Delta Table Azure Databricks Learning: Delta Lake: How to re-order columns of a delta table ================================================================================= Re-ordering tables columns is one of the most common requirement in database and data warehousing concepts. It is also improving performance in Databricks delta lake. To know more about it watch this video https://youtu.be/cnWmN8T6E9I #Deltalake #DataSkipping #DeltaSkipping #ReorderDeltaColumns # RepositionDeltaColumns"  
[YouTube Link](https://youtube.com/watch?v=cnWmN8T6E9I)  2023-09-05T12:30Z 38.4K followers, [----] engagements


"113. Databricks PySpark Spark Reader: Skip Specific Range of Records While Reading CSV File Azure Databricks Learning: Spark Reader: Skip Specific Range of Records While Reading CSV File ================================================================================= Processing CSV files in Spark and Databricks is one of the very frequently seen scenario. While reading CSV data we come across requirement of skipping range of records in middle of CSV file in certain use cases. I have explained that requirement in this video To get more understanding watch this video"  
[YouTube Link](https://youtube.com/watch?v=j-X48mxuIHo)  2023-07-13T12:30Z 38.4K followers, [----] engagements


"112. Databricks Pyspark Spark Reader: Skip First N Records While Reading CSV File Azure Databricks Learning: Spark Reader: Skip First N Records While Reading CSV File ================================================================================= Processing CSV files in Spark and Databricks is one of the very frequently seen scenario. While reading CSV data we come across requirement of skipping first few records in certain usecases. I have explained that requirement in this video To get more understanding watch this video #SparkCSVReader #SparkCSVSkipRows"  
[YouTube Link](https://youtube.com/watch?v=IAwbbqio3OU)  2023-07-12T12:30Z 38.4K followers, [----] engagements


"111. Databricks Pyspark SQL Coding Interview: Exchange Seats of Students Azure Databricks Learning: Coding Interview Exercise: Pyspark and Spark SQL ================================================================================= Coding exercises are very common in most of the Bigdata interviews. It is important to develop coding skills before appearing for Spark/Databricks interviews. In this video I have explained a coding scenario to exchange the seats of students in a class. This is Leet Code SQL Exercise number [---]. This is also one of the FAANG company question Google Microsoft Amazon"  
[YouTube Link](https://youtube.com/watch?v=JBiqD3umB6s)  2023-07-11T12:30Z 38.4K followers, 10.9K engagements


"110. Databricks Pyspark Spark Reader: Reading Fixed Length Text File Azure Databricks Learning: Spark Reader: Reading Fixed Length Text File ======================================================================== Spark Reader is one of basic and widely used concept in Spark development. In this video I have covered how to read text file and create Dataframe out of it. I used a fixed length text file for this exercise and splitted the fixed length records into multiple columns To get thorough understanding of this concept watch this video #SparkReader"  
[YouTube Link](https://youtube.com/watch?v=9k1yltr1T6E)  2023-07-07T12:30Z 38.4K followers, [----] engagements


"109. Databricks Pyspark Coding Interview Question: Pyspark and Spark SQL Azure Databricks Learning: Coding Interview Exercise: Pyspark and Spark SQL ================================================================================= Coding exercises are very common in most of the Bigdata interviews. It is important to develop coding skills before appearing for Spark/Databricks interviews. In this video I have explained a coding scenario to find out start and end date of data buckets. To get more understanding watch this video #CodingInterviewQuestion #ApacheSparkInterview #SparkCodingExercise"  
[YouTube Link](https://youtube.com/watch?v=VsM7cqa2fBs)  2023-07-06T12:30Z 38.4K followers, 30.3K engagements


"108. Databricks Pyspark Window Function: First and Last Azure Databricks Learning: Pyspark Development: Window Function: First and Last ================================================================================= Both First and Last are Pyspark Window functions used to idetify first and last value of a column for each window of a dataset. To get more understanding of the functionalities of these [--] transformation watch this video #DatabricksWindowTransformation#PysparkWindowTransformation #PysparkFirst #PysparkLast#SparkWindowFunctions#SparkDevelopment#DatabricksDevelopment"  
[YouTube Link](https://youtube.com/watch?v=Q0uJK13yxzg)  2023-07-05T12:30Z 38.4K followers, [----] engagements


"107. Databricks Pyspark Transformation: Subtract vs ExceptAll Azure Databricks Learning: Pyspark Development: Transformation: Subtract vs ExceptAll ================================================================================= Both subtract and exceptAll are important PySpark transformations that can be used for data cleaning and data analysis. They are used to identify the difference between [--] dataframes. To get more understanding of the functionalities of these [--] transformation and difference between these two watch this video"  
[YouTube Link](https://youtube.com/watch?v=ihZYockAxd0)  2023-07-04T12:30Z 38.4K followers, [----] engagements


"106.DatabricksPysparkAutomationReal Time Project:DataType Issue When Writing to Azure Synapse/SQL Azure Databricks Learning: Pyspark Development: Real Time Project: DataType Issue While Writing Into Azure Synapse/SQL ================================================================================= How to handle data type mismatch issue between databricks and azure data warehouse while writing dataframe into ADW This video provides an automated solution approach to handle data type mismatch issue between databricks and azure data warehouse."  
[YouTube Link](https://youtube.com/watch?v=6fnLvoDnmZQ)  2023-07-03T12:30Z 38.4K followers, [----] engagements


"105. Databricks Pyspark Pyspark Development: Spark/Databricks Interview Question Series - V Azure Databricks Learning: Pyspark Development: Spark/Databricks Interview Question Series - V ================================================================================= Are you learning Spark/Databricks to become Bigdata Engineer Are you preparing for an Interview for the role of Spark/Databricks data engineer Pyspark development is core area for any spark/databricks projects and we can expect more questions from this topic. Follow this video to get list of questions on Pyspark Development"  
[YouTube Link](https://youtube.com/watch?v=d_feL0mj-5E)  2023-07-01T13:30Z 38.4K followers, [----] engagements


"104. Databricks Pyspark Pyspark Development: Spark/Databricks Interview Question Series - IV Azure Databricks Learning: Pyspark Development: Spark/Databricks Interview Question Series - IV ================================================================================= Are you learning Spark/Databricks to become Bigdata Engineer Are you preparing for an Interview for the role of Spark/Databricks data engineer Pyspark development is core area for any spark/databricks projects and we can expect more questions from this topic. Follow this video to get list of questions on Pyspark Development"  
[YouTube Link](https://youtube.com/watch?v=2dzgLQ3khTk)  2023-06-24T12:30Z 38.4K followers, [----] engagements


"103. Databricks Pyspark Delta Lake: Spark/Databricks Interview Question Series - III Azure Databricks Learning: Delta Lake: Spark/Databricks Interview Question Series - III ================================================================================= Are you learning Spark/Databricks to become Bigdata Engineer Are you preparing for an Interview for the role of Spark/Databricks data engineer Delta Lake is one of the modern Lakehouse concept which is used on most of bigdata projects. This is also one of the major topic to crack interviews. Follow this video to get list of questions on Delta"  
[YouTube Link](https://youtube.com/watch?v=VK9ZA688GuM)  2023-06-13T12:30Z 38.4K followers, 11.8K engagements


"102. Databricks Pyspark Performance Optimization: Spark/Databricks Interview Question Series - II Azure Databricks Learning: Performance Optimization: Spark/Databricks Interview Question Series - II ================================================================================= Are you learning Spark/Databricks to become Bigdata Engineer Are you preparing for an Interview for the role of Spark/Databricks data engineer Performance optimization is one of the constant topic in all interview calls. Follow this video to get list of questions on performance optimization concepts along with"  
[YouTube Link](https://youtube.com/watch?v=FTDecDchRkw)  2023-05-30T12:30Z 38.4K followers, 14.2K engagements


"101. Databricks Pyspark Core/Architecture: Spark/Databricks Interview Question Series - I Azure Databricks Learning: Spark/Databricks Interview Questions on Core/Architecture Concepts -I ================================================================================= Are you learning Spark/Databricks to get a job Are you preparing for Interview for the role of Spark/Databricks data engineer Follow this video to get list of questions on spark/Databricks core concepts along with directions to give answer in the interview #SparkInterviewQuestions"  
[YouTube Link](https://youtube.com/watch?v=AUvnhHeHriA)  2023-05-18T15:32Z 38.4K followers, 12.9K engagements


"100. Databricks Pyspark Spark Architecture: Internals of Partition Creation Demystified Azure Databricks Learning: Spark Architecture: Internals of Partition Creation Demystified ================================================================================= How partitions are created within spark environment out of external storage system How number of partitions are decided for given set of input files/folders Partition is key to any big data platform. It is important for every developer/architect to understand the internal working mechanism of partition creation. But it has always been"  
[YouTube Link](https://youtube.com/watch?v=A80o9WGXK_I)  2023-04-12T13:36Z 38.4K followers, 17.2K engagements


"99. Databricks Pyspark Real Time Use Case: Generate Test Data - Array_Repeat() Azure Databricks Learning: Real Time Use Case: Generate Test Data - Array_Repeat() ================================================================================= What is the functionality of Array_Repeat How to generate the test data in Databricks development Generating test data is one of the key task in testing phase for performance testing and stress testing. I have explained the process of generating huge volume of data in short time using array_repeat function To get through understanding of this concept"  
[YouTube Link](https://youtube.com/watch?v=qWcl_fbIQ-U)  2023-04-05T15:23Z 38.4K followers, [----] engagements


"98. Databricks Pyspark Interview Question: Pyspark VS Pandas Azure Databricks Learning: Interview Question: Pyspark VS Pandas =================================================================== What are the key differences between pyspark and pandas Pandas and Pyspark both are used in bigdata development for data processing data analytics and data science. Though both are python libraries there are many key differences between both of them in terms of handling data volume processing speed etc. To get through understanding of this concept please watch this video #PandasVSPyspark"  
[YouTube Link](https://youtube.com/watch?v=ZRORmf3DZto)  2023-03-30T16:04Z 38.4K followers, [----] engagements


"97. Databricks Pyspark Data Security: Enforcing Column Level Encryption Azure Databricks Learning: Data Security: Enforcing Column Level Encryption =================================================================== How to implement data security features in Databricks Development Data security is of utmost importance in Databricks for several reasons such as Compliance with regulations Maintaining trust with customers Preventing data breaches etc. In this video I have explained how to enforce column-level encryption in databricks development To get through understanding of this concept"  
[YouTube Link](https://youtube.com/watch?v=nu5_dOKAJcg)  2023-03-23T13:05Z 38.4K followers, 14.3K engagements


"96. Databricks Pyspark Real Time Scenario Schema Comparison Azure Databricks Learning: Schema Comparison ============================================ How to compare schemas of different dataframes and make it same through automated way Schema related operations are very common in Databricks development. Have explained different real time scenarios of schema related operations in this video To get through understanding of this concept please watch this video #DatabricksSchemaComparison #DatabricksStructType#DatabricksMapType#PysparkStructType#PysparkMapType"  
[YouTube Link](https://youtube.com/watch?v=BtUFleFkXMM)  2023-02-03T14:15Z 38.4K followers, 11.1K engagements


"95. Databricks Pyspark Schema Different Methods of Schema Definition Struct Type vs Struct Field : https://youtu.be/Ff2XvNfsAn8 Struct Type vs Map Type: https://youtu.be/wI-nqFPW580 Reading CSV Files: https://youtu.be/7mfKuo_Ng_Q Azure Databricks Learning: Different Methods of Schema Definition ========================================================= What are the different methods of defining schema in databricks using pyspark Schema definition is on of the basic and most commonly used operation in Databricks development. Have explained different methods of defining schema in this video To"  
[YouTube Link](https://youtube.com/watch?v=ZMa7tlXlg-0)  2023-02-02T15:53Z 38.4K followers, [----] engagements


"94. Databricks Pyspark Interview Question Schema Definition: Struct Type vs Map Type Azure Databricks Learning: Interview Question Schema Definition: Struct Type vs Map Type ================================================================================ What is the difference between pyspark methods StructType and Maptype StructType and Maptype both are used to define structure of a nested field dataframe. But both are used for different use cases. I have explained the difference in this video To get through understanding of this concept please watch this video"  
[YouTube Link](https://youtube.com/watch?v=wI-nqFPW580)  2022-12-20T12:31Z 38.4K followers, [----] engagements


"93. Databricks Pyspark Interview Question Schema Definition: Struct Type vs Struct Field Azure Databricks Learning: Interview Question Schema Definition: Struct Type vs Struct Field ================================================================================ What is the difference between pyspark methods StructType and StructField StructType and StructField both are used to define structure of a dataframe. To get through understanding of this concept please watch this video #DatabricksStructType#DatabricksStructField#PysparkStructType#PysparkStructField"  
[YouTube Link](https://youtube.com/watch?v=Ff2XvNfsAn8)  2022-12-16T12:30Z 38.4K followers, [----] engagements


"92. Databricks Pyspark Interview Question Performance Optimization: Select vs WithColumn Azure Databricks Learning: Interview Question Performance Optimization: Select vs WithColumn ================================================================================ What is the difference between pyspark functions select and withcolumn Select and withcolumn both are used to add new columns to existing dataframe. But select outperforms withcolumn. The reason behind this difference is explained in this video. To get through understanding of this concept please watch this video"  
[YouTube Link](https://youtube.com/watch?v=-Q7xNTPcEFA)  2022-12-14T12:30Z 38.4K followers, 12.3K engagements


"91. Databricks Pyspark Interview Question Handlining Duplicate Data: DropDuplicates vs Distinct Azure Databricks Learning: Interview Question - Handlining Duplicate Data: DropDuplicates vs Distinct ================================================================================ How to eliminate duplicate in dataframe What is the difference between Distinct and DropDuplicates Understanding different mechanisms of handling duplicate records is essential in databricks development. Also undertstanding the difference between distinct and dropDuplicates is important to clear the interview. To get"  
[YouTube Link](https://youtube.com/watch?v=CmUNa_USfdU)  2022-12-12T12:30Z 38.4K followers, 10.8K engagements


"90. Databricks Pyspark Interview Question: Read Excel File with Multiple Sheets Azure Databricks Learning: Interview Question: Read Excel File with Multiple Sheets ================================================================================ How to create dataframe reading multiple excel sheets Though creating dataframe by reading excel sheets is not very common still there are certain scenarios where we need to read excel data. Reading data from all excel sheets is bit challenging as there is no direct solution. I have created an automated solution in this video for that requirement To"  
[YouTube Link](https://youtube.com/watch?v=h-FrNWj1NXo)  2022-12-07T12:30Z 38.4K followers, 14.2K engagements


"89. Databricks Pyspark Notebook Scheduling through Event Based Trigger using Azure Data Factory Azure Databricks Learning: Notebook Scheduling through Event Based Trigger using ADF ================================================================================ How to schedule Databricks Notebook through event based trigger using Azure Data Factory ADF plays pivotal role in scheduling various ETL pipelines through ADF activities. Azure databricks notebook can be scheduled through notebook activity in Azure data factory. Event based trigger creates job scheduling as soon as file arrives. To"  
[YouTube Link](https://youtube.com/watch?v=dP-ZXgxx5TY)  2022-12-06T12:30Z 38.4K followers, 13.3K engagements


"88. Databricks Pyspark Notebook Scheduling through Schedule Based Trigger using Azure Data Factory Azure Databricks Learning: Notebook Scheduling through Schedule Based Trigger using ADF ================================================================================ How to schedule Databricks Notebook through schedule based trigger using Azure Data Factory ADF plays pivotal role in scheduling various ETL pipelines through ADF activities. Azure databricks notebook can be scheduled through notebook activity in Azure data factory. Schedule based trigger creates job scheduling at regular"  
[YouTube Link](https://youtube.com/watch?v=8bYSXzAnntE)  2022-12-05T12:30Z 38.4K followers, [----] engagements


"87. Databricks Pyspark Real Time Project: ETL Pipeline Integrating ADF ASQL ADLS Key Vault Azure Databricks Learning: Real Time Project:ETL Pipeline Integrating Databricks ADF ASQL ADLS and Key Vault ================================================================================ How to develop ETL Pipeline Integrating Databricks ADF ASQL ADLS and Key Vault This tutorial explains the real time project scenario of ETL pipeline development of integrating Azure Databricks with ADF ASQL ADLS and Key Vault To get through understanding of this concept please watch this video"  
[YouTube Link](https://youtube.com/watch?v=dxxXWe4gNTo)  2022-12-04T12:30Z 38.4K followers, 23K engagements


"86. Databricks Pyspark Notebook Activity in Azure Data Factory with Output Parameter Azure Databricks Learning: Execute Azure Databricks Notebook through Azure Data Factory with Output Parameter ================================================================================ How can we execute the databricks notebook from Azure data factory with output parameter This tutorial explains the process of running databricks notebook through notebook activity in Azure Data Factory with output parameter To get through understanding of this concept please watch this video #DatabricksOutputParameter"  
[YouTube Link](https://youtube.com/watch?v=g9Hm6XtCLUo)  2022-12-03T12:30Z 38.4K followers, 10.5K engagements


"85. Databricks Pyspark Notebook Activity in Azure Data Factory with Input Parameter Azure Databricks Learning: Execute Azure Databricks Notebook through Azure Data Factory with Input Paramters ================================================================================ How can we execute the Databricks notebook from Azure data factory with input parameters This tutorial explains the process of running databricks notebook through notebook activity in Azure Data Factory with Input parameters To get through understanding of this concept please watch this video #DatabricksWidgets"  
[YouTube Link](https://youtube.com/watch?v=ldVgPhjaB7w)  2022-12-02T12:30Z 38.4K followers, 16.3K engagements


"84. Databricks Pyspark Azure Data Factory + Azure Databricks: Execute Notebook Via ADF Azure Databricks Learning: Execute Azure Databricks Notebook through Azure Data Factory ================================================================================ How can we execute the databricks notebook from Azure data factory Azure data factory is known for its scheduling features so it is chosen as a scheduler for most of the projects. To execute the notebook from ADF Azure Data Factory provides Notebook activity through which notebook can be invoked. To get through understanding of this concept"  
[YouTube Link](https://youtube.com/watch?v=fLZ3b9uiPEI)  2022-12-01T12:30Z 38.4K followers, 20.1K engagements

Limited data mode. Full metrics available with subscription: lunarcrush.com/pricing