ksh pipe <SOLVED>

About writing shell scripts and making the most of your shell
Forum rules
Topics in this forum are automatically closed 6 months after creation.
Locked
klong915

ksh pipe <SOLVED>

Post by klong915 »

Hi,
I'm not sure that I'm using this post correctly but will learn. I would like to implement a simple pipe script. look for something (ls something) then pipe that into sed and have results go to same file name in a tmp directory. Problem is ls may have several hits.
What variable is being piped from ls to sed , or do you recomend doing this another way?

ls <some_entry> | `sed -e 's/something/else' >/tmp/

thanks,
keith
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 2 times in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
User avatar
xenopeek
Level 25
Level 25
Posts: 29509
Joined: Wed Jul 06, 2011 3:58 am

Re: ksh pipe

Post by xenopeek »

Can you explain a bit more what you want to do because the ls piped into sed, with parameter to load a script while you don't give which script to load, doesn't make much sense.

Do you want to replace text in the output of ls, or do you want to replace text in the files that ls found?

If you want to replace text in the output of ls, you would do that as:

Code: Select all

ls <pattern of filenames to search for> | sed 's/something/else/' > /tmp/yourtempfile
That replaces one match per line. If you want to replace all matches on a line, add "g" after the final "/" in the sed statement. As in:

Code: Select all

ls <pattern of filenames to search for> | sed 's/something/else/g' > /tmp/yourtempfile
Image
klong915

Re: ksh pipe

Post by klong915 »

Hi,
thanks for getting back. Yes I'm aware of what you've written, and was not clear what I was trying to do "simply".
Goal is to sort out files via `ls` pipe them into `sed` and have sed write out the alterations made to a tmp folder
containing the same file name. for instance: `ls` on current folder will show "a b c tmp" a,b,c are files tmp is folder.
so `ls [a-z]` renders "a b c"
files a,b,c contain 1 line all equal to "123".
sed -e 's/2/7/' [the piped in file names from ls [a-z]] >tmp/[piped in file names from ls [a-z]]
so when complete dir "try"
contains "a b c" files having one line to read "173"
I thought syntax should look something like:
ls [a-z] | `sed -e's/2/7/' "HOW DO I SPECIFY THE FILE REDIRECT?" want >tmp/a, >tmp/b, >tmp/c `

Hope this makes more sense.
Thanks,
Keith
User avatar
xenopeek
Level 25
Level 25
Posts: 29509
Joined: Wed Jul 06, 2011 3:58 am

Re: ksh pipe

Post by xenopeek »

It's not going to work that way, because after the redirect you have no access to the names of the files. I'd go about this another way. You could first copy the files you need to destination directory like with cp [abc] /tmp and then run sed -i 's/2/7/' * from /tmp to update the files in place. Then there is no need for redirect.

You can do it with one statement also, and the absolute safest way to do it is like this:

Code: Select all

while IFS= read -d $'\0' -r file ; do
	sed 's/2/7/' "$file" > "/tmp/$(basename \"$file\")" 
done < <(find . -name '[abc]' -print0)
This will work correctly with files that have spaces or newlines as part of their names. See http://stackoverflow.com/a/7039579 for description of how this works.
Image
klong915

Re: ksh pipe

Post by klong915 »

Hi,
hope your patient. What you wrote:
while IFS= read -d $'\0' -r file ; do
sed 's/2/7/' "$file" > "/tmp/$(basename \"$file\")"
done < <(find . -name '[abc]' -print0)

ok, if I enter this directly nothing happens. I was not sure about where exactly you wanted the files {a,b,c} to reside so I tried with them in current directory and then in current-dir and tmp-dir. Both cases the script exits and nothing else is shown or happens.
on the last line you use command substitution with the find command finding {a,b,c} . This gets imported to the while loop.
the second line does the sed thing to $file, what defines $file, also where did "basename" come from?
the first line invokes while loop based on IFS and reads a file deliminated by null; again what defines file?

Sorry if this too slow, I did look things up but I'm missing a key parameter.

Thanks,
Keith
User avatar
xenopeek
Level 25
Level 25
Posts: 29509
Joined: Wed Jul 06, 2011 3:58 am

Re: ksh pipe

Post by xenopeek »

The while loop reads input from the results of the find. find prints each filename, terminated with a \0 (null) character. Hence the while loop sets IFS to empty, to avoid doing any filename splitting. basename is a standard Linux command.

I tried this command out, running it from a directory where I had files called "a", "b", and "c", each with contents "123". It worked fine. It does assume your current shell is bash. If you're using a different shell... Are you indeed using ksh?
Image
klong915

Re: ksh pipe

Post by klong915 »

Hi,
Yes I'm using ksh, but I also tried same in bash; they appeared to behave the same (nothing happens, no message no files).
Thanks for your persistence, I'm too am trying to figure out where I am here. Maybe there is a shell problem. I recently downloaded ubuntu 13.1 64 bit. They default to bash also but I went though menu script to establish ksh and for all I can gather it was successful. Both ksh and bash allow the programming you entered I think. This is ksh93 which allows command substitution (last entry). I will leave ksh again and restart in bash. Last time I only entered /bin/bash in the current ksh window. Thanks again for your concern.
Keith
User avatar
xenopeek
Level 25
Level 25
Posts: 29509
Joined: Wed Jul 06, 2011 3:58 am

Re: ksh pipe

Post by xenopeek »

The command wouldn't print out anything, though you can add that if you want:

Code: Select all

while IFS= read -d $'\0' -r file ; do
   echo "about to process $file"
   sed 's/2/7/' "$file" > "/tmp/$(basename \"$file\")"
done < <(find . -name '[abc]' -print0)
Image
klong915

Re: ksh pipe

Post by klong915 »

Yes, I understand; what I meant by no messages was there weren't any including any error messages.
OK, booted in bash, echo $SHELL->/bin/bash
cut/paste your script in xterm window in a directory containing {a,b,c,tmp}
return prompt is instant but tmp doesn't get populated with altered {a,b,c}
User avatar
xenopeek
Level 25
Level 25
Posts: 29509
Joined: Wed Jul 06, 2011 3:58 am

Re: ksh pipe

Post by xenopeek »

To be clear, because you have that directory tmp there, files are written to /tmp ... not to the subdirectory tmp of your current directory.
Image
klong915

Re: ksh pipe

Post by klong915 »

Yes, good. I was looking locally ./tmp .
in bash only the script works. I edited line: sed 's/2/7/' "$file" > "/tmp/$(basename \"$file\")" to sed 's/2/7/' "$file" > "/tmp/$(basename $file)" .

Great thanks for you patients and help,
Keith

P.S. I should close this correct? I don't see means to do so at moment will look about.
User avatar
xenopeek
Level 25
Level 25
Posts: 29509
Joined: Wed Jul 06, 2011 3:58 am

Re: ksh pipe

Post by xenopeek »

You can mark a topic closed by clicking the edit button in the top right of your first post here, and adding the word <SOLVED> to the subject. That updates the subject of the topic and shows others it's been solved.

PS. I sent you a PM about your username, and got a reply back from you which you deleted before I got a chance to read it. PM me again if you wanted a change of username.
Image
Locked

Return to “Scripts & Bash”