How to grep and cut numbers from a file and sum them The 2019 Stack Overflow Developer Survey...
A pet rabbit called Belle
Python - Fishing Simulator
Working through the single responsibility principle (SRP) in Python when calls are expensive
How did the audience guess the pentatonic scale in Bobby McFerrin's presentation?
Is this wall load bearing? Blueprints and photos attached
Who or what is the being for whom Being is a question for Heidegger?
Is there a writing software that you can sort scenes like slides in PowerPoint?
Change bounding box of math glyphs in LuaTeX
Derivation tree not rendering
Hopping to infinity along a string of digits
Does the AirPods case need to be around while listening via an iOS Device?
When did F become S in typeography, and why?
Was credit for the black hole image misattributed?
Simulating Exploding Dice
ELI5: Why do they say that Israel would have been the fourth country to land a spacecraft on the Moon and why do they call it low cost?
What are these Gizmos at Izaña Atmospheric Research Center in Spain?
What do you call a plan that's an alternative plan in case your initial plan fails?
Can withdrawing asylum be illegal?
Match Roman Numerals
Why can't wing-mounted spoilers be used to steepen approaches?
How does this infinite series simplify to an integral?
How do you keep chess fun when your opponent constantly beats you?
Keeping a retro style to sci-fi spaceships?
Is it ethical to upload a automatically generated paper to a non peer-reviewed site as part of a larger research?
How to grep and cut numbers from a file and sum them
The 2019 Stack Overflow Developer Survey Results Are In
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
2019 Community Moderator Election Results
Why I closed the “Why is Kali so hard” questionGrep alias - line numbers unless it's in a pipelineIssue with real-time log inspecting piping tail, grep and cutcopy every line from a text file that contains a number greater than 5000How to unbuffer cut?Grep to filter and show only the beginning of a lineTail Grep - Print surrounding lines until pattern is matchedGrep and filter IP from text fileawk/sed/grep: Printing all lines matching a string and all lines with tabs after these linesGrep only numbers, not the alphanumeric entriesHow do I add numbers from two txt files with Bash?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
I have a log file. For every line with a specific number, I want to sum the last number of those lines. To grep and cut is no problem but I don't know how to sum the numbers. I tried some solutions from StackExchange but didn't get them to work in my case.
This is what I have so far:
grep "30201" logfile.txt | cut -f6 -d "|"
30201 are the lines I'm looking for.
I want to sum the last numbers 650, 1389 and 945
The logfile.txt
Jan 09 2016|09:15:17|30201|1|SL02|650
Jan 09 2016|09:15:18|43097|1|SL01|945
Jan 09 2016|09:15:19|28774|2|SB03|1389
Jan 09 2016|09:16:21|00788|1|SL02|650
Jan 09 2016|09:17:25|03361|3|SL01|945
Jan 09 2016|09:17:33|08385|1|SL02|650
Jan 09 2016|09:18:43|10234|1|SL01|945
Jan 09 2016|09:21:55|00788|1|SL02|650
Jan 09 2016|09:24:43|03361|3|SB03|1389
Jan 09 2016|09:26:01|30201|1|SB03|1389
Jan 09 2016|09:26:21|28774|2|SL02|650
Jan 09 2016|09:26:25|00788|1|SL02|650
Jan 09 2016|09:27:21|28774|2|SL02|650
Jan 09 2016|09:29:32|30201|1|SL01|945
Jan 09 2016|09:30:12|34032|1|SB03|1389
Jan 09 2016|09:30:15|08767|3|SL02|650
text-processing grep logs cut numeric-data
add a comment |
I have a log file. For every line with a specific number, I want to sum the last number of those lines. To grep and cut is no problem but I don't know how to sum the numbers. I tried some solutions from StackExchange but didn't get them to work in my case.
This is what I have so far:
grep "30201" logfile.txt | cut -f6 -d "|"
30201 are the lines I'm looking for.
I want to sum the last numbers 650, 1389 and 945
The logfile.txt
Jan 09 2016|09:15:17|30201|1|SL02|650
Jan 09 2016|09:15:18|43097|1|SL01|945
Jan 09 2016|09:15:19|28774|2|SB03|1389
Jan 09 2016|09:16:21|00788|1|SL02|650
Jan 09 2016|09:17:25|03361|3|SL01|945
Jan 09 2016|09:17:33|08385|1|SL02|650
Jan 09 2016|09:18:43|10234|1|SL01|945
Jan 09 2016|09:21:55|00788|1|SL02|650
Jan 09 2016|09:24:43|03361|3|SB03|1389
Jan 09 2016|09:26:01|30201|1|SB03|1389
Jan 09 2016|09:26:21|28774|2|SL02|650
Jan 09 2016|09:26:25|00788|1|SL02|650
Jan 09 2016|09:27:21|28774|2|SL02|650
Jan 09 2016|09:29:32|30201|1|SL01|945
Jan 09 2016|09:30:12|34032|1|SB03|1389
Jan 09 2016|09:30:15|08767|3|SL02|650
text-processing grep logs cut numeric-data
add a comment |
I have a log file. For every line with a specific number, I want to sum the last number of those lines. To grep and cut is no problem but I don't know how to sum the numbers. I tried some solutions from StackExchange but didn't get them to work in my case.
This is what I have so far:
grep "30201" logfile.txt | cut -f6 -d "|"
30201 are the lines I'm looking for.
I want to sum the last numbers 650, 1389 and 945
The logfile.txt
Jan 09 2016|09:15:17|30201|1|SL02|650
Jan 09 2016|09:15:18|43097|1|SL01|945
Jan 09 2016|09:15:19|28774|2|SB03|1389
Jan 09 2016|09:16:21|00788|1|SL02|650
Jan 09 2016|09:17:25|03361|3|SL01|945
Jan 09 2016|09:17:33|08385|1|SL02|650
Jan 09 2016|09:18:43|10234|1|SL01|945
Jan 09 2016|09:21:55|00788|1|SL02|650
Jan 09 2016|09:24:43|03361|3|SB03|1389
Jan 09 2016|09:26:01|30201|1|SB03|1389
Jan 09 2016|09:26:21|28774|2|SL02|650
Jan 09 2016|09:26:25|00788|1|SL02|650
Jan 09 2016|09:27:21|28774|2|SL02|650
Jan 09 2016|09:29:32|30201|1|SL01|945
Jan 09 2016|09:30:12|34032|1|SB03|1389
Jan 09 2016|09:30:15|08767|3|SL02|650
text-processing grep logs cut numeric-data
I have a log file. For every line with a specific number, I want to sum the last number of those lines. To grep and cut is no problem but I don't know how to sum the numbers. I tried some solutions from StackExchange but didn't get them to work in my case.
This is what I have so far:
grep "30201" logfile.txt | cut -f6 -d "|"
30201 are the lines I'm looking for.
I want to sum the last numbers 650, 1389 and 945
The logfile.txt
Jan 09 2016|09:15:17|30201|1|SL02|650
Jan 09 2016|09:15:18|43097|1|SL01|945
Jan 09 2016|09:15:19|28774|2|SB03|1389
Jan 09 2016|09:16:21|00788|1|SL02|650
Jan 09 2016|09:17:25|03361|3|SL01|945
Jan 09 2016|09:17:33|08385|1|SL02|650
Jan 09 2016|09:18:43|10234|1|SL01|945
Jan 09 2016|09:21:55|00788|1|SL02|650
Jan 09 2016|09:24:43|03361|3|SB03|1389
Jan 09 2016|09:26:01|30201|1|SB03|1389
Jan 09 2016|09:26:21|28774|2|SL02|650
Jan 09 2016|09:26:25|00788|1|SL02|650
Jan 09 2016|09:27:21|28774|2|SL02|650
Jan 09 2016|09:29:32|30201|1|SL01|945
Jan 09 2016|09:30:12|34032|1|SB03|1389
Jan 09 2016|09:30:15|08767|3|SL02|650
text-processing grep logs cut numeric-data
text-processing grep logs cut numeric-data
edited 28 mins ago
Jeff Schaller♦
45k1164147
45k1164147
asked 12 hours ago
YungScholarYungScholar
233
233
add a comment |
add a comment |
4 Answers
4
active
oldest
votes
You can take help from paste
to serialize the numbers in a format suitable for bc
to do the addition:
% grep "30201" logfile.txt | cut -f6 -d "|"
650
1389
945
% grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+
650+1389+945
% grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+ | bc
2984
If you have grep
with PCRE, you can do it with grep
alone using postive lookbehind:
% grep -Po '|30201|.*|Kd+' logfile.txt | cut -f6 -d "|" | paste -sd+ | bc
2984
With awk
alone:
% awk -F'|' '$3 == 30201 {sum+=$NF}; END{print sum}' logfile.txt
2984
-F'|'
sets the field separator as|
$3 == 30201 {sum+=$NF}
adds up the last field's values if the third field is30201
END{print sum}
prints thesum
at theEND
Thanks!grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+
dind't work for me, I got this errors(standard_in) 1: illegal character: ^M
. But the solution withgrep -Po '|30201|.*|Kd+'
works. This is great!
– YungScholar
12 hours ago
2
Note that thegrep
solution does not care in what column the number is found, or whether the number is just a substring of a longer number. Theawk
solution is safer in this respect. Thegrep
solution could be improved by first cutting and the matching the number at the start of the line (followed by|
) with proper anchoring.
– Kusalananda♦
10 hours ago
add a comment |
Bash solution.
#!/bin/bash
pa=0 ; s=0 ;
while read a b ; do
if [ "$a" == "$pa" ] ; then
s=$(($s+$b)) ;
else
if [ "$pa" != 0 ] ; then
echo $pa $s ;
fi ;
pa=$a ; s=$b ;
fi ;
done < <(cat j.txt | awk -F'|' '{printf("%s %sn",$3,$6)}' | sort -n)
echo $pa $s
Init Previous A and SUM
Cut down the input to fields 3 and 6 and sort them by number
Loop as long as field 3 stays the same, add field 6 to the SUM
if field 3 changes but the Previous A is not 0, output the Previous A and the SUM and reinit Previous A to a and SUM to last field 6 read.
Output last Previous A and SUM.
Output of the given input:
00788 1950
03361 2334
08385 650
08767 650
10234 945
28774 2689
30201 2984
34032 1389
43097 945
Given you are using awk anyway to select the third and sixth columns, you should go the extra steps and sum things inside awk. This would give you something likeawk -F'|' '{s[$3]+=$6}END{ for (i in s) { print i, s[i] }}' | sort
- GNU awk has a builtinasort
which could also be used rather than an external sort.
– icarus
4 hours ago
add a comment |
There is nothing really wrong with your grep and cut command. You could make it more robust by using "|30201|" as the search pattern. The issue then is dealing with the output.
Using bash:
#!/bin/bash
# get the output as a bash array and add the elements
nums=( $(grep "|30201|" logfile.txt | cut -f6 -d "|") )
total=0
for i in ${!nums[@]}
do
total=$(($total+${nums[i]}))
done
echo $total
add a comment |
One little tool I keep around I call sumcol
#!/bin/sh
# Icarus Sparry. Free for any use.
C=${1:?"missing required column number"}
shift
awk '{s+=$'"$C"'} END { print s }' "$@"
which adds up the whitespace delimited column you provide. Whilst I would write (as @heemayl does)
awk -F'|' '$3 == 30201 {s+=$6} END{ print s}' logfile.txt
for the OP's problem, he could use
grep "30201" logfile.txt | cut -f6 -d "|" | sumcol 1
or
grep "30201" logfile.txt | tr "| " " _" | sumcol 6
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f512250%2fhow-to-grep-and-cut-numbers-from-a-file-and-sum-them%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can take help from paste
to serialize the numbers in a format suitable for bc
to do the addition:
% grep "30201" logfile.txt | cut -f6 -d "|"
650
1389
945
% grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+
650+1389+945
% grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+ | bc
2984
If you have grep
with PCRE, you can do it with grep
alone using postive lookbehind:
% grep -Po '|30201|.*|Kd+' logfile.txt | cut -f6 -d "|" | paste -sd+ | bc
2984
With awk
alone:
% awk -F'|' '$3 == 30201 {sum+=$NF}; END{print sum}' logfile.txt
2984
-F'|'
sets the field separator as|
$3 == 30201 {sum+=$NF}
adds up the last field's values if the third field is30201
END{print sum}
prints thesum
at theEND
Thanks!grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+
dind't work for me, I got this errors(standard_in) 1: illegal character: ^M
. But the solution withgrep -Po '|30201|.*|Kd+'
works. This is great!
– YungScholar
12 hours ago
2
Note that thegrep
solution does not care in what column the number is found, or whether the number is just a substring of a longer number. Theawk
solution is safer in this respect. Thegrep
solution could be improved by first cutting and the matching the number at the start of the line (followed by|
) with proper anchoring.
– Kusalananda♦
10 hours ago
add a comment |
You can take help from paste
to serialize the numbers in a format suitable for bc
to do the addition:
% grep "30201" logfile.txt | cut -f6 -d "|"
650
1389
945
% grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+
650+1389+945
% grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+ | bc
2984
If you have grep
with PCRE, you can do it with grep
alone using postive lookbehind:
% grep -Po '|30201|.*|Kd+' logfile.txt | cut -f6 -d "|" | paste -sd+ | bc
2984
With awk
alone:
% awk -F'|' '$3 == 30201 {sum+=$NF}; END{print sum}' logfile.txt
2984
-F'|'
sets the field separator as|
$3 == 30201 {sum+=$NF}
adds up the last field's values if the third field is30201
END{print sum}
prints thesum
at theEND
Thanks!grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+
dind't work for me, I got this errors(standard_in) 1: illegal character: ^M
. But the solution withgrep -Po '|30201|.*|Kd+'
works. This is great!
– YungScholar
12 hours ago
2
Note that thegrep
solution does not care in what column the number is found, or whether the number is just a substring of a longer number. Theawk
solution is safer in this respect. Thegrep
solution could be improved by first cutting and the matching the number at the start of the line (followed by|
) with proper anchoring.
– Kusalananda♦
10 hours ago
add a comment |
You can take help from paste
to serialize the numbers in a format suitable for bc
to do the addition:
% grep "30201" logfile.txt | cut -f6 -d "|"
650
1389
945
% grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+
650+1389+945
% grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+ | bc
2984
If you have grep
with PCRE, you can do it with grep
alone using postive lookbehind:
% grep -Po '|30201|.*|Kd+' logfile.txt | cut -f6 -d "|" | paste -sd+ | bc
2984
With awk
alone:
% awk -F'|' '$3 == 30201 {sum+=$NF}; END{print sum}' logfile.txt
2984
-F'|'
sets the field separator as|
$3 == 30201 {sum+=$NF}
adds up the last field's values if the third field is30201
END{print sum}
prints thesum
at theEND
You can take help from paste
to serialize the numbers in a format suitable for bc
to do the addition:
% grep "30201" logfile.txt | cut -f6 -d "|"
650
1389
945
% grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+
650+1389+945
% grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+ | bc
2984
If you have grep
with PCRE, you can do it with grep
alone using postive lookbehind:
% grep -Po '|30201|.*|Kd+' logfile.txt | cut -f6 -d "|" | paste -sd+ | bc
2984
With awk
alone:
% awk -F'|' '$3 == 30201 {sum+=$NF}; END{print sum}' logfile.txt
2984
-F'|'
sets the field separator as|
$3 == 30201 {sum+=$NF}
adds up the last field's values if the third field is30201
END{print sum}
prints thesum
at theEND
edited 12 hours ago
answered 12 hours ago
heemaylheemayl
36.4k378108
36.4k378108
Thanks!grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+
dind't work for me, I got this errors(standard_in) 1: illegal character: ^M
. But the solution withgrep -Po '|30201|.*|Kd+'
works. This is great!
– YungScholar
12 hours ago
2
Note that thegrep
solution does not care in what column the number is found, or whether the number is just a substring of a longer number. Theawk
solution is safer in this respect. Thegrep
solution could be improved by first cutting and the matching the number at the start of the line (followed by|
) with proper anchoring.
– Kusalananda♦
10 hours ago
add a comment |
Thanks!grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+
dind't work for me, I got this errors(standard_in) 1: illegal character: ^M
. But the solution withgrep -Po '|30201|.*|Kd+'
works. This is great!
– YungScholar
12 hours ago
2
Note that thegrep
solution does not care in what column the number is found, or whether the number is just a substring of a longer number. Theawk
solution is safer in this respect. Thegrep
solution could be improved by first cutting and the matching the number at the start of the line (followed by|
) with proper anchoring.
– Kusalananda♦
10 hours ago
Thanks!
grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+
dind't work for me, I got this errors (standard_in) 1: illegal character: ^M
. But the solution with grep -Po '|30201|.*|Kd+'
works. This is great!– YungScholar
12 hours ago
Thanks!
grep "30201" logfile.txt | cut -f6 -d "|" | paste -sd+
dind't work for me, I got this errors (standard_in) 1: illegal character: ^M
. But the solution with grep -Po '|30201|.*|Kd+'
works. This is great!– YungScholar
12 hours ago
2
2
Note that the
grep
solution does not care in what column the number is found, or whether the number is just a substring of a longer number. The awk
solution is safer in this respect. The grep
solution could be improved by first cutting and the matching the number at the start of the line (followed by |
) with proper anchoring.– Kusalananda♦
10 hours ago
Note that the
grep
solution does not care in what column the number is found, or whether the number is just a substring of a longer number. The awk
solution is safer in this respect. The grep
solution could be improved by first cutting and the matching the number at the start of the line (followed by |
) with proper anchoring.– Kusalananda♦
10 hours ago
add a comment |
Bash solution.
#!/bin/bash
pa=0 ; s=0 ;
while read a b ; do
if [ "$a" == "$pa" ] ; then
s=$(($s+$b)) ;
else
if [ "$pa" != 0 ] ; then
echo $pa $s ;
fi ;
pa=$a ; s=$b ;
fi ;
done < <(cat j.txt | awk -F'|' '{printf("%s %sn",$3,$6)}' | sort -n)
echo $pa $s
Init Previous A and SUM
Cut down the input to fields 3 and 6 and sort them by number
Loop as long as field 3 stays the same, add field 6 to the SUM
if field 3 changes but the Previous A is not 0, output the Previous A and the SUM and reinit Previous A to a and SUM to last field 6 read.
Output last Previous A and SUM.
Output of the given input:
00788 1950
03361 2334
08385 650
08767 650
10234 945
28774 2689
30201 2984
34032 1389
43097 945
Given you are using awk anyway to select the third and sixth columns, you should go the extra steps and sum things inside awk. This would give you something likeawk -F'|' '{s[$3]+=$6}END{ for (i in s) { print i, s[i] }}' | sort
- GNU awk has a builtinasort
which could also be used rather than an external sort.
– icarus
4 hours ago
add a comment |
Bash solution.
#!/bin/bash
pa=0 ; s=0 ;
while read a b ; do
if [ "$a" == "$pa" ] ; then
s=$(($s+$b)) ;
else
if [ "$pa" != 0 ] ; then
echo $pa $s ;
fi ;
pa=$a ; s=$b ;
fi ;
done < <(cat j.txt | awk -F'|' '{printf("%s %sn",$3,$6)}' | sort -n)
echo $pa $s
Init Previous A and SUM
Cut down the input to fields 3 and 6 and sort them by number
Loop as long as field 3 stays the same, add field 6 to the SUM
if field 3 changes but the Previous A is not 0, output the Previous A and the SUM and reinit Previous A to a and SUM to last field 6 read.
Output last Previous A and SUM.
Output of the given input:
00788 1950
03361 2334
08385 650
08767 650
10234 945
28774 2689
30201 2984
34032 1389
43097 945
Given you are using awk anyway to select the third and sixth columns, you should go the extra steps and sum things inside awk. This would give you something likeawk -F'|' '{s[$3]+=$6}END{ for (i in s) { print i, s[i] }}' | sort
- GNU awk has a builtinasort
which could also be used rather than an external sort.
– icarus
4 hours ago
add a comment |
Bash solution.
#!/bin/bash
pa=0 ; s=0 ;
while read a b ; do
if [ "$a" == "$pa" ] ; then
s=$(($s+$b)) ;
else
if [ "$pa" != 0 ] ; then
echo $pa $s ;
fi ;
pa=$a ; s=$b ;
fi ;
done < <(cat j.txt | awk -F'|' '{printf("%s %sn",$3,$6)}' | sort -n)
echo $pa $s
Init Previous A and SUM
Cut down the input to fields 3 and 6 and sort them by number
Loop as long as field 3 stays the same, add field 6 to the SUM
if field 3 changes but the Previous A is not 0, output the Previous A and the SUM and reinit Previous A to a and SUM to last field 6 read.
Output last Previous A and SUM.
Output of the given input:
00788 1950
03361 2334
08385 650
08767 650
10234 945
28774 2689
30201 2984
34032 1389
43097 945
Bash solution.
#!/bin/bash
pa=0 ; s=0 ;
while read a b ; do
if [ "$a" == "$pa" ] ; then
s=$(($s+$b)) ;
else
if [ "$pa" != 0 ] ; then
echo $pa $s ;
fi ;
pa=$a ; s=$b ;
fi ;
done < <(cat j.txt | awk -F'|' '{printf("%s %sn",$3,$6)}' | sort -n)
echo $pa $s
Init Previous A and SUM
Cut down the input to fields 3 and 6 and sort them by number
Loop as long as field 3 stays the same, add field 6 to the SUM
if field 3 changes but the Previous A is not 0, output the Previous A and the SUM and reinit Previous A to a and SUM to last field 6 read.
Output last Previous A and SUM.
Output of the given input:
00788 1950
03361 2334
08385 650
08767 650
10234 945
28774 2689
30201 2984
34032 1389
43097 945
answered 11 hours ago
JdeHaanJdeHaan
359214
359214
Given you are using awk anyway to select the third and sixth columns, you should go the extra steps and sum things inside awk. This would give you something likeawk -F'|' '{s[$3]+=$6}END{ for (i in s) { print i, s[i] }}' | sort
- GNU awk has a builtinasort
which could also be used rather than an external sort.
– icarus
4 hours ago
add a comment |
Given you are using awk anyway to select the third and sixth columns, you should go the extra steps and sum things inside awk. This would give you something likeawk -F'|' '{s[$3]+=$6}END{ for (i in s) { print i, s[i] }}' | sort
- GNU awk has a builtinasort
which could also be used rather than an external sort.
– icarus
4 hours ago
Given you are using awk anyway to select the third and sixth columns, you should go the extra steps and sum things inside awk. This would give you something like
awk -F'|' '{s[$3]+=$6}END{ for (i in s) { print i, s[i] }}' | sort
- GNU awk has a builtin asort
which could also be used rather than an external sort.– icarus
4 hours ago
Given you are using awk anyway to select the third and sixth columns, you should go the extra steps and sum things inside awk. This would give you something like
awk -F'|' '{s[$3]+=$6}END{ for (i in s) { print i, s[i] }}' | sort
- GNU awk has a builtin asort
which could also be used rather than an external sort.– icarus
4 hours ago
add a comment |
There is nothing really wrong with your grep and cut command. You could make it more robust by using "|30201|" as the search pattern. The issue then is dealing with the output.
Using bash:
#!/bin/bash
# get the output as a bash array and add the elements
nums=( $(grep "|30201|" logfile.txt | cut -f6 -d "|") )
total=0
for i in ${!nums[@]}
do
total=$(($total+${nums[i]}))
done
echo $total
add a comment |
There is nothing really wrong with your grep and cut command. You could make it more robust by using "|30201|" as the search pattern. The issue then is dealing with the output.
Using bash:
#!/bin/bash
# get the output as a bash array and add the elements
nums=( $(grep "|30201|" logfile.txt | cut -f6 -d "|") )
total=0
for i in ${!nums[@]}
do
total=$(($total+${nums[i]}))
done
echo $total
add a comment |
There is nothing really wrong with your grep and cut command. You could make it more robust by using "|30201|" as the search pattern. The issue then is dealing with the output.
Using bash:
#!/bin/bash
# get the output as a bash array and add the elements
nums=( $(grep "|30201|" logfile.txt | cut -f6 -d "|") )
total=0
for i in ${!nums[@]}
do
total=$(($total+${nums[i]}))
done
echo $total
There is nothing really wrong with your grep and cut command. You could make it more robust by using "|30201|" as the search pattern. The issue then is dealing with the output.
Using bash:
#!/bin/bash
# get the output as a bash array and add the elements
nums=( $(grep "|30201|" logfile.txt | cut -f6 -d "|") )
total=0
for i in ${!nums[@]}
do
total=$(($total+${nums[i]}))
done
echo $total
edited 7 hours ago
answered 7 hours ago
WastrelWastrel
11
11
add a comment |
add a comment |
One little tool I keep around I call sumcol
#!/bin/sh
# Icarus Sparry. Free for any use.
C=${1:?"missing required column number"}
shift
awk '{s+=$'"$C"'} END { print s }' "$@"
which adds up the whitespace delimited column you provide. Whilst I would write (as @heemayl does)
awk -F'|' '$3 == 30201 {s+=$6} END{ print s}' logfile.txt
for the OP's problem, he could use
grep "30201" logfile.txt | cut -f6 -d "|" | sumcol 1
or
grep "30201" logfile.txt | tr "| " " _" | sumcol 6
add a comment |
One little tool I keep around I call sumcol
#!/bin/sh
# Icarus Sparry. Free for any use.
C=${1:?"missing required column number"}
shift
awk '{s+=$'"$C"'} END { print s }' "$@"
which adds up the whitespace delimited column you provide. Whilst I would write (as @heemayl does)
awk -F'|' '$3 == 30201 {s+=$6} END{ print s}' logfile.txt
for the OP's problem, he could use
grep "30201" logfile.txt | cut -f6 -d "|" | sumcol 1
or
grep "30201" logfile.txt | tr "| " " _" | sumcol 6
add a comment |
One little tool I keep around I call sumcol
#!/bin/sh
# Icarus Sparry. Free for any use.
C=${1:?"missing required column number"}
shift
awk '{s+=$'"$C"'} END { print s }' "$@"
which adds up the whitespace delimited column you provide. Whilst I would write (as @heemayl does)
awk -F'|' '$3 == 30201 {s+=$6} END{ print s}' logfile.txt
for the OP's problem, he could use
grep "30201" logfile.txt | cut -f6 -d "|" | sumcol 1
or
grep "30201" logfile.txt | tr "| " " _" | sumcol 6
One little tool I keep around I call sumcol
#!/bin/sh
# Icarus Sparry. Free for any use.
C=${1:?"missing required column number"}
shift
awk '{s+=$'"$C"'} END { print s }' "$@"
which adds up the whitespace delimited column you provide. Whilst I would write (as @heemayl does)
awk -F'|' '$3 == 30201 {s+=$6} END{ print s}' logfile.txt
for the OP's problem, he could use
grep "30201" logfile.txt | cut -f6 -d "|" | sumcol 1
or
grep "30201" logfile.txt | tr "| " " _" | sumcol 6
answered 4 hours ago
icarusicarus
6,21611231
6,21611231
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f512250%2fhow-to-grep-and-cut-numbers-from-a-file-and-sum-them%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown