create account

AWK, My Number 1 Data Manipulation Tool by lifeof.abdul

View this thread on: hive.blogpeakd.comecency.com
· @lifeof.abdul · (edited)
$5.85
AWK, My Number 1 Data Manipulation Tool
<div class="text-justify">
<center> https://images.ecency.com/DQmVNnv6yrVBwYpJVz456jr9tqfGF1SdUSxv9oCyE7oWW2t/awk.png
<sub> made with canva </sub></center>

Good day, fellow developers, I haven't shared a post here in a long time. Today, I would like to talk about awk, an incredible tool for all UNIX based systems. Since learning about awk, I've had less stress when manipulating any type of data because awk was designed to make it easier. 

If you're looking for a tool to take your data manipulation and analysis to the next level, you should look into Awk. Awk is a powerful tool for shell scripting and data manipulation that allows you to quickly and easily manipulate data on the command line. It's incredibly versatile and can be used to write simple scripts to parse data from text files and to perform complex data manipulation operations. 


I used to  perform most of my data manipulation stuff with Excel, but I just realized awk does it better and with ease. I now use awk for all of my data manipulation tasks, including text searching, filtering, and modification. Regardless of file size, awk handles data manipulation and analysis operations efficiently. With Awk, you can quickly extract data from a text file, transform it into a more useful format, and analyze it without having to type out lengthy commands or write complex scripts.

**In the following demonstration, I used awk to convert a text file containing comma-separated values of student details into a more meaningful format.**

You can see in the screenshot below that I have a text file with values separated by commas. You can see that the data is almost not meaningful and can't be used for a presentation. Assuming you want to make a presentation with this data.

<center>![](https://images.ecency.com/DQmPB5gXzd3ESeQFK4CMsqVwS4FCQQEiaiiXef55QEDu8AY/students.png)
 <sub> Screenshot from my pc </sub></center>

To manipulate this data with awk, I can either write the whole script on the command line and execute it or create a reusable script that I can easily reuse for my subsequent data manipulation on similar data.

Here I will be writing the script to manipulate the data in a text editor and saving it as a .awk file, which I can always reuse. The screenshot below shows the script used to convert the data shown below into a nicely formatted data set with additional calculations.An awk script file contains three parts. The first part is "BEGIN", which contains the heading section of your resulting data. The second part contains code to manipulate the data. And the final part, "END," displays whatever you want after manipulation.Check out the script below:

```BEGIN {
    FS=","
    print " Display of Students Results   "
    
    # printing header row
    printf "%-10s\t%-20s\t%-10s\t%-15s\t%-10s\t%-10s\n", "Stdt ID", "Stdt Name", "Class Mrk(max 70)", "Exm Rslt(max 50)","Final Results(%)", "Status"
    printf "----------------------------------------------------------\n"
}
# process data rows
NR > 1 {
    studentID = $1
    studentName= $2
    CSM = $3
    examMark= $4
    finalResults = ((CSM+examMark)/120)*100
    if (finalResults < 50){
      status = "Failed"}
    else {status="Passed"}
{
 # printing the formatted row
    printf "%-10s\t%-30s\t%-20s\t%-10s\t%-20s\t%-10s\n", studentID ,studentName, CSM, examMark, int(finalResults), status
    # update total number of students
    num_ofStudents += 1
} 
}
# print totals if data rows exist
END { 
        printf "-----------------------------------------------------------\n"
        print "                                    "
        printf "     Total Students:\t%-10d\n", num_ofStudents
}
```
In the above code, we have all the three parts in our script. To run the script, we open the terminal and run this command ```awk -f your_script.awk your_data.awk```. To save the results in a new txt file say results.txt we add it like this ```awk -f your_script.awk your_data.awk < results.txt``` The screenshot below  shows how the result will look that when ran on the terminal. 

<center> ![](https://images.ecency.com/DQmVm2Rq1roHnpbZLHhyY2XApfpcuU7hKmqvdWkL326Ye3h/studsu.png)
<sub> Screenshot from my pc </sub></center>

You can see how nicely awk has formatted our data with ease. As if that weren't enough, Awk is also incredibly easy to learn and use. Even if you're new to Linux and shell scripting, you can quickly learn how to use Awk to perform data manipulation and analysis operations with ease. Plus, there are plenty of online resources and tutorials available to help get you up and running with Awk in no time. 

</div>


<center>

------------------------------------

```I hope you found this useful. I am Abdul-Salam Issahaku, from Ghana. I am a student web developer and a blockchain enthusiast. I blog about technology, life and investments. You can follow me to be part of my Hive family. Thank you for your time.```

---------------------------------------------- 
![](https://images.ecency.com/DQmPfoCi5Q4WSqQ8kK7ZQFQKnsfxECUbxezFEwfRtigMqnq/hive.gif)
<sub>GIF by @doze  </sub>




</center>
👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 134 others
properties (23)
authorlifeof.abdul
permlinkawk-my-number-1-data
categoryhive-169321
json_metadata"{"image":["https://images.ecency.com/DQmVNnv6yrVBwYpJVz456jr9tqfGF1SdUSxv9oCyE7oWW2t/awk.png","https://images.ecency.com/DQmPB5gXzd3ESeQFK4CMsqVwS4FCQQEiaiiXef55QEDu8AY/students.png","https://images.ecency.com/DQmVm2Rq1roHnpbZLHhyY2XApfpcuU7hKmqvdWkL326Ye3h/studsu.png","https://images.ecency.com/DQmPfoCi5Q4WSqQ8kK7ZQFQKnsfxECUbxezFEwfRtigMqnq/hive.gif"],"tags":["hive-169321","ocd","hivedevs","hiveghana","neoxian","proofofbrain","awk","linux","programming","coding"],"description":"made with [canva](canva.com) Good day, fellow developers, I haven't shared a post here in a long time. Today, I would like to talk about awk, an incredible tool for all UNIX based systems. Since learning","app":"ecency/3.0.31-vision","format":"markdown+html","image_ratios":[1.7777777777777777,1.608294930875576,2.237762237762238,1.8181818181818181]}"
created2023-03-23 23:12:03
last_update2023-03-23 23:13:00
depth0
children2
last_payout2023-03-30 23:12:03
cashout_time1969-12-31 23:59:59
total_payout_value2.940 HBD
curator_payout_value2.905 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length5,087
author_reputation26,928,429,948,423
root_title"AWK, My Number 1 Data Manipulation Tool"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id121,911,388
net_rshares11,295,202,914,837
author_curate_reward""
vote details (198)
@hivebuzz ·
Congratulations @lifeof.abdul! You have completed the following achievement on the Hive blockchain And have been rewarded with New badge(s)

<table><tr><td><img src="https://images.hive.blog/60x70/http://hivebuzz.me/@lifeof.abdul/replies.png?202303240503"></td><td>You got more than 300 replies.<br>Your next target is to reach 400 replies.</td></tr>
</table>

<sub>_You can view your badges on [your board](https://hivebuzz.me/@lifeof.abdul) and compare yourself to others in the [Ranking](https://hivebuzz.me/ranking)_</sub>
<sub>_If you no longer want to receive notifications, reply to this comment with the word_ `STOP`</sub>



**Check out our last posts:**
<table><tr><td><a href="/hive-139531/@hivebuzz/proposal-2324"><img src="https://images.hive.blog/64x128/https://i.imgur.com/RNIZ1N6.png"></a></td><td><a href="/hive-139531/@hivebuzz/proposal-2324">The Hive Gamification Proposal</a></td></tr></table>
properties (22)
authorhivebuzz
permlinknotify-lifeofabdul-20230324t052153
categoryhive-169321
json_metadata{"image":["http://hivebuzz.me/notify.t6.png"]}
created2023-03-24 05:21:54
last_update2023-03-24 05:21:54
depth1
children0
last_payout2023-03-31 05:21:54
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length913
author_reputation367,873,730,322,291
root_title"AWK, My Number 1 Data Manipulation Tool"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id121,918,864
net_rshares0
@stemsocial ·
re-lifeofabdul-awk-my-number-1-data-20230324t032802681z
<div class='text-justify'> <div class='pull-left'>
 <img src='https://stem.openhive.network/images/stemsocialsupport7.png'> </div>

Thanks for your contribution to the <a href='/trending/hive-196387'>STEMsocial community</a>. Feel free to join us on <a href='https://discord.gg/9c7pKVD'>discord</a> to get to know the rest of us!

Please consider delegating to the @stemsocial account (85% of the curation rewards are returned).

You may also include @stemsocial as a beneficiary of the rewards of this post to get a stronger support.&nbsp;<br />&nbsp;<br />
</div>
properties (22)
authorstemsocial
permlinkre-lifeofabdul-awk-my-number-1-data-20230324t032802681z
categoryhive-169321
json_metadata{"app":"STEMsocial"}
created2023-03-24 03:28:03
last_update2023-03-24 03:28:03
depth1
children0
last_payout2023-03-31 03:28:03
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length565
author_reputation22,460,334,324,555
root_title"AWK, My Number 1 Data Manipulation Tool"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id121,916,957
net_rshares0