ksh pipe <SOLVED>

Forum rules
Before you post please read how to get help

ksh pipe <SOLVED>

Postby klong915@gmail.com on Wed Feb 05, 2014 7:39 am

Hi,
I'm not sure that I'm using this post correctly but will learn. I would like to implement a simple pipe script. look for something (ls something) then pipe that into sed and have results go to same file name in a tmp directory. Problem is ls may have several hits.
What variable is being piped from ls to sed , or do you recomend doing this another way?

ls <some_entry> | `sed -e 's/something/else' >/tmp/

thanks,
keith
Last edited by klong915@gmail.com on Sat Feb 08, 2014 2:38 am, edited 1 time in total.
klong915@gmail.com
Level 1
Level 1
 
Posts: 18
Joined: Wed Feb 05, 2014 7:22 am

Linux Mint is funded by ads and donations.
 

Re: ksh pipe

Postby xenopeek on Wed Feb 05, 2014 2:37 pm

Can you explain a bit more what you want to do because the ls piped into sed, with parameter to load a script while you don't give which script to load, doesn't make much sense.

Do you want to replace text in the output of ls, or do you want to replace text in the files that ls found?

If you want to replace text in the output of ls, you would do that as:
Code: Select all
ls <pattern of filenames to search for> | sed 's/something/else/' > /tmp/yourtempfile

That replaces one match per line. If you want to replace all matches on a line, add "g" after the final "/" in the sed statement. As in:
Code: Select all
ls <pattern of filenames to search for> | sed 's/something/else/g' > /tmp/yourtempfile
User avatar
xenopeek
Level 21
Level 21
 
Posts: 15437
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Re: ksh pipe

Postby klong915@gmail.com on Wed Feb 05, 2014 10:51 pm

Hi,
thanks for getting back. Yes I'm aware of what you've written, and was not clear what I was trying to do "simply".
Goal is to sort out files via `ls` pipe them into `sed` and have sed write out the alterations made to a tmp folder
containing the same file name. for instance: `ls` on current folder will show "a b c tmp" a,b,c are files tmp is folder.
so `ls [a-z]` renders "a b c"
files a,b,c contain 1 line all equal to "123".
sed -e 's/2/7/' [the piped in file names from ls [a-z]] >tmp/[piped in file names from ls [a-z]]
so when complete dir "try"
contains "a b c" files having one line to read "173"
I thought syntax should look something like:
ls [a-z] | `sed -e's/2/7/' "HOW DO I SPECIFY THE FILE REDIRECT?" want >tmp/a, >tmp/b, >tmp/c `

Hope this makes more sense.
Thanks,
Keith
klong915@gmail.com
Level 1
Level 1
 
Posts: 18
Joined: Wed Feb 05, 2014 7:22 am

Re: ksh pipe

Postby xenopeek on Thu Feb 06, 2014 3:11 am

It's not going to work that way, because after the redirect you have no access to the names of the files. I'd go about this another way. You could first copy the files you need to destination directory like with cp [abc] /tmp and then run sed -i 's/2/7/' * from /tmp to update the files in place. Then there is no need for redirect.

You can do it with one statement also, and the absolute safest way to do it is like this:
Code: Select all
while IFS= read -d $'\0' -r file ; do
   sed 's/2/7/' "$file" > "/tmp/$(basename \"$file\")"
done < <(find . -name '[abc]' -print0)

This will work correctly with files that have spaces or newlines as part of their names. See http://stackoverflow.com/a/7039579 for description of how this works.
User avatar
xenopeek
Level 21
Level 21
 
Posts: 15437
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Re: ksh pipe

Postby klong915@gmail.com on Thu Feb 06, 2014 8:02 pm

Hi,
hope your patient. What you wrote:
while IFS= read -d $'\0' -r file ; do
sed 's/2/7/' "$file" > "/tmp/$(basename \"$file\")"
done < <(find . -name '[abc]' -print0)

ok, if I enter this directly nothing happens. I was not sure about where exactly you wanted the files {a,b,c} to reside so I tried with them in current directory and then in current-dir and tmp-dir. Both cases the script exits and nothing else is shown or happens.
on the last line you use command substitution with the find command finding {a,b,c} . This gets imported to the while loop.
the second line does the sed thing to $file, what defines $file, also where did "basename" come from?
the first line invokes while loop based on IFS and reads a file deliminated by null; again what defines file?

Sorry if this too slow, I did look things up but I'm missing a key parameter.

Thanks,
Keith
klong915@gmail.com
Level 1
Level 1
 
Posts: 18
Joined: Wed Feb 05, 2014 7:22 am

Re: ksh pipe

Postby xenopeek on Fri Feb 07, 2014 3:25 am

The while loop reads input from the results of the find. find prints each filename, terminated with a \0 (null) character. Hence the while loop sets IFS to empty, to avoid doing any filename splitting. basename is a standard Linux command.

I tried this command out, running it from a directory where I had files called "a", "b", and "c", each with contents "123". It worked fine. It does assume your current shell is bash. If you're using a different shell... Are you indeed using ksh?
User avatar
xenopeek
Level 21
Level 21
 
Posts: 15437
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Re: ksh pipe

Postby klong915@gmail.com on Fri Feb 07, 2014 4:20 am

Hi,
Yes I'm using ksh, but I also tried same in bash; they appeared to behave the same (nothing happens, no message no files).
Thanks for your persistence, I'm too am trying to figure out where I am here. Maybe there is a shell problem. I recently downloaded ubuntu 13.1 64 bit. They default to bash also but I went though menu script to establish ksh and for all I can gather it was successful. Both ksh and bash allow the programming you entered I think. This is ksh93 which allows command substitution (last entry). I will leave ksh again and restart in bash. Last time I only entered /bin/bash in the current ksh window. Thanks again for your concern.
Keith
klong915@gmail.com
Level 1
Level 1
 
Posts: 18
Joined: Wed Feb 05, 2014 7:22 am

Re: ksh pipe

Postby xenopeek on Fri Feb 07, 2014 4:24 am

The command wouldn't print out anything, though you can add that if you want:
Code: Select all
while IFS= read -d $'\0' -r file ; do
   echo "about to process $file"
   sed 's/2/7/' "$file" > "/tmp/$(basename \"$file\")"
done < <(find . -name '[abc]' -print0)
User avatar
xenopeek
Level 21
Level 21
 
Posts: 15437
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Re: ksh pipe

Postby klong915@gmail.com on Fri Feb 07, 2014 4:37 am

Yes, I understand; what I meant by no messages was there weren't any including any error messages.
OK, booted in bash, echo $SHELL->/bin/bash
cut/paste your script in xterm window in a directory containing {a,b,c,tmp}
return prompt is instant but tmp doesn't get populated with altered {a,b,c}
klong915@gmail.com
Level 1
Level 1
 
Posts: 18
Joined: Wed Feb 05, 2014 7:22 am

Re: ksh pipe

Postby xenopeek on Fri Feb 07, 2014 4:57 am

To be clear, because you have that directory tmp there, files are written to /tmp ... not to the subdirectory tmp of your current directory.
User avatar
xenopeek
Level 21
Level 21
 
Posts: 15437
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Re: ksh pipe

Postby klong915@gmail.com on Fri Feb 07, 2014 5:41 am

Yes, good. I was looking locally ./tmp .
in bash only the script works. I edited line: sed 's/2/7/' "$file" > "/tmp/$(basename \"$file\")" to sed 's/2/7/' "$file" > "/tmp/$(basename $file)" .

Great thanks for you patients and help,
Keith

P.S. I should close this correct? I don't see means to do so at moment will look about.
klong915@gmail.com
Level 1
Level 1
 
Posts: 18
Joined: Wed Feb 05, 2014 7:22 am

Re: ksh pipe

Postby xenopeek on Fri Feb 07, 2014 6:33 am

You can mark a topic closed by clicking the edit button in the top right of your first post here, and adding the word <SOLVED> to the subject. That updates the subject of the topic and shows others it's been solved.

PS. I sent you a PM about your username, and got a reply back from you which you deleted before I got a chance to read it. PM me again if you wanted a change of username.
User avatar
xenopeek
Level 21
Level 21
 
Posts: 15437
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Linux Mint is funded by ads and donations.
 

Return to Scripts & Bash

Who is online

Users browsing this forum: No registered users and 2 guests