I have a dataframe called DF1 like below.
DF1:
srcColumnZ|srcCoulmnY|srcCoulmnR|
+---------+----------+----------+
|John |Non Hf |New york |
|Steav |Non Hf |Mumbai |
|Ram |HF |Boston |
And also having one list of map with source to target column mapping like below.
List(Map(targetColumn -> columnNameX, sourceColumn -> List(srcColumnX, srcColumnY, srcColumnZ, srcColumnP, srcColumnQ, srcColumnR)), Map(targetColumn -> columnNameY, sourceColumn -> List(srcColumnY)), Map(targetColumn -> columnNameZ, selectvalue -> 5))
I wanted to create a dataframe based on the above list of Map and in that data frame I need columnNameX, columnNameY, columnNameZ as a column(according to above list) and the value of these column will be based on sourceColumn i.e. if sourceColumn is present like List(srcColumnX, srcColumnY, srcColumnZ, srcColumnP, srcColumnQ, srcColumnR)) then it will check all the column one by one in DF1 and whenever 1st column will match it will move all the values of that column into target column and same for next target column. And in case selectvalue present instead of source column it will hardcode that value into entire column. ie: in above list for target column(columnNameZ) selectvalue is present 5
Below is the expected output.
columnNameX|columnNameY|columnNameZ|
+----------+-----------+-----------+
|John |Non Hf |5 |
|Steav |Non Hf |5 |
|Ram |HF |5 |