How to insert contents of a file (or a line of text) at a given location inside hundreds of text files?

[Andy Burnelli]

How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?

It seems that vpngate.net vpn configuration files now require a username
(of vpn) and a password (of vpn), which isn't a big deal for most people
who don't use automation, as they'd just type the login/password in when
asked by their openvpn client graphical user interface.

But I change the Time Zone & VPN server every few minutes via batch files.

It's easy to _manually (aurgh!) add at line 103 the "password.txt" line,
but what might you think is an easy way to add that line to _every_ file?

Here's what I do manually (which, admittedly, is a PITA albeit simple):

1. cd directory-containing-hundreds-of-openvpn-config-files
2. vim *.ovpn
3. :103
4. ~r include.txt
5. :write (abbreviated as :w)
6. :next (abbreviated as :n)

1. That manual process starts in a directory of many files
2. and then edits, one by one, each of a few hundred *.ovpn files;
3. and then it jumps to line 103 (it's always in the same spot);
4. and then I include the file, which contains the following line only:
auth-user-pass C:\\path\\config\\passwd.txt
5. and then I write the *.ovpn file;
6. and then I skip to the next *.ovpn file.

Where the contents of the c:\path\config\passwd.txt file is two lines:
vpn
vpn
The first being the username & the latter being the password as described
in the <http://vpngate.net> home page (this is a brand new requirement).

If I were to simply append the line to the _end_ of the *.ovpn files,
instead of inserting thatline at line 103, I could use a Windows command
line foreach loop to append the "include.txt" file to the *.ovpn file:
for %f in (*.ovpn) do type include.txt >> %f
But how can I _insert+ that one-line include.txt file at line 103?

How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?

[Jeff Barnett]

Post by Andy Burnelli
How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?
It seems that vpngate.net vpn configuration files now require a username
(of vpn) and a password (of vpn), which isn't a big deal for most people
who don't use automation, as they'd just type the login/password in when
asked by their openvpn client graphical user interface.
But I change the Time Zone & VPN server every few minutes via batch files.
It's easy to _manually (aurgh!) add at line 103 the "password.txt" line,
but what might you think is an easy way to add that line to _every_ file?
1. cd directory-containing-hundreds-of-openvpn-config-files
2. vim *.ovpn
3. :103
4. ~r include.txt
5. :write (abbreviated as :w)
6. :next  (abbreviated as :n)
1. That manual process starts in a directory of many files
2. and then edits, one by one, each of a few hundred *.ovpn files;
3. and then it jumps to line 103 (it's always in the same spot); 4. and
   auth-user-pass C:\\path\\config\\passwd.txt
5. and then I write the *.ovpn file;
6. and then I skip to the next *.ovpn file.
   vpn
   vpn The first being the username & the latter being the password as
described
in the <http://vpngate.net> home page (this is a brand new requirement).
If I were to simply append the line to the _end_ of the *.ovpn files,
instead of inserting thatline at line 103, I could use a Windows command
for %f in (*.ovpn) do type include.txt >> %f
But how can I _insert+ that one-line include.txt file at line 103?
How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?

What about writing a few-line C program? Or one

[Paul]

Post by Andy Burnelli
How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?
It seems that vpngate.net vpn configuration files now require a username
(of vpn) and a password (of vpn), which isn't a big deal for most people
who don't use automation, as they'd just type the login/password in when
asked by their openvpn client graphical user interface.
But I change the Time Zone & VPN server every few minutes via batch files.
It's easy to _manually (aurgh!) add at line 103 the "password.txt" line, but what might you think is an easy way to add that line to _every_ file?
1. cd directory-containing-hundreds-of-openvpn-config-files
2. vim *.ovpn
3. :103
4. ~r include.txt
5. :write (abbreviated as :w)
6. :next  (abbreviated as :n)
1. That manual process starts in a directory of many files
2. and then edits, one by one, each of a few hundred *.ovpn files;
    auth-user-pass C:\\path\\config\\passwd.txt
5. and then I write the *.ovpn file;
6. and then I skip to the next *.ovpn file.
    vpn
    vpn The first being the username & the latter being the password as described
in the <http://vpngate.net> home page (this is a brand new requirement).
If I were to simply append the line to the _end_ of the *.ovpn files, instead of inserting thatline at line 103, I could use a Windows command
for %f in (*.ovpn) do type include.txt >> %f
But how can I _insert+ that one-line include.txt file at line 103?
How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?

What about writing a few-line C program? Or one of a language you prefer.

There are things like sed, AWK (GAWK), and Perl
for textual processing.

GAWK comes with its own book ("gawk.pdf") by Arnold Robbins.
"GAWK: Effective AWK Programming", Several editions. Gawk3
is available via GnuWin32. Gawk4 is available via WSL/WSLg.
For standard tasks, either version would do.

With gawk you can code up a state machine (to do your
macro processing and keep track of what you're doing
and what trigger you are looking for at the moment).
Gawk can open multiple files at once -- you can code up
a diff() for yourself for example. Gawk can call system
executables ("dir") and pipe the contents for processing.

Perl can do all of those things too.

With C, there would be a temptation to write some
low level text handlers, that a textual processor
would already have. Maybe you can find a library for C
to act as an assistant, so it's not an impossibility.

If all you knew was C, you would use C.

And those are all procedural languages.

*******

You can also use a macro recorder, to record a sequence
and do stuff. This is also programming, because understanding
the limitations of the macro recorder is just as complex
as learning a programming language.

*******

The first step in a project like this, is breaking down
the process into pseudocode, so that anyone can follow it.
If you can explain to people what you're doing, successfully,
then programming it is a doddle. If your specification is
fuzzy, your program will also be fuzzy.

Some of the little programs I've written, I will have like
a dozen test files. Very simple files with only a few lines
in them. I feed those to the program, to iron out the bugs
in the concept. This is also necessary when your program
has to work with "real world" items you yourself did not invent.

Paul

[Andy Burnelli]

Post by Paul
The first step in a project like this, is breaking down
the process into pseudocode, so that anyone can follow it.
If you can explain to people what you're doing, successfully,
then programming it is a doddle. If your specification is
fuzzy, your program will also be fuzzy.

Hi Paul,

(This is written in explanatory-pseudocode so others can easily follow it.)

I am familiar (albeit rather unduly rusty) with Unix shell programming
having spent most of my grad school & Silicon Valley time on the 'nix's.

And I wrote a tutorial, long ago when it first came out, for installing WSL
on Windows in five minutes, along with Cygwin tutorials in long years past.

But I'm trying to see if I can do this either with a vim macro or better
yet, in the Windows command line shell, where luckily I have a real world
example anyone can see which is easily reproduced by doing the following
(which I provide so that others can _benefit_ from zillions of free VPNs!).

A. Go to http://vpngate.net
B. In the 7th column named "OpenVPN" are the free VPN openvpn config files.
C. Press on a few of them (see why below) to download them to your system.

Here is an example link to one of them, but these links change constantly:
<https://www.vpngate.net/en/do_openvpn.aspx?fqdn=vpn371239765.opengw.net&ip=180.198.24.136&tcp=1519&udp=1381&sid=1671465724963&hid=17464287>
Simplified as this free fully qualified domain name openvpn config file:
<https://www.vpngate.net/en/do_openvpn.aspx?fqdn=vpn371239765.opengw.net>

That will allow you to download your choice of free openvpn config files:
a. A UDP .ovpn config file which uses a port 53 DNS DDNS hostname lookup:
filename = "vpn371239765.opengw.net (UDP 1381)"
b. A TCP .ovpn config file which uses a port 53 DNS DDNS hostname lookup:
filename = "vpn371239765.opengw.net (TCP 1519)"
c. A UDP .ovpn file which skips the port 53 DNS lookup to foil big brother:
filename = "180.198.24.136 (UDP 1381)"
d. A TCP .ovpn file which skips the port 53 DNS lookup to foil big brother:
filename = "180.198.24.136 (TCP 1519)"

Up until very recently, that file is the only thing you needed to do in
order to programmatically switch VPN servers every few minutes on Windows.
Pseudocode...
set configcmd=C:\path-to\openvpn\bin\openvpn.exe
set configdir=C:\path-to\openvpn\config\
set configfile=(that file above chosen randomly programmatically)
cd %configdir%
%configcmd% %configdir%%configfile%

I could never get this batch file to run "as" admin without _being_ admin,
so you need to right-click on that batch file to execute with administrator
privileges (which are needed in order to change the routing table).

Notice _all_ eight thousand of these free openvpn config files are of the
same format, where line 103 in every one is the following line every time:
#auth-user-pass

What I want to do is change that (or simply add this below that) to:
auth-user-pass C:\\path-to\\config\\username_password.txt
Where the C:\path-to\config\username_password.txt file contains two lines:
vpn
vpn
Which are the (apparently) brand new requirements for adding to openvpn
username
password
(Note: Up until recently, this username/password requirement wasn't there!)

I have found that it works to just append (or prepend) that line, so I'm
actually already well into my workaround but the problem set remains valid.
*How do you _insert_ a line into a file using the Windows command line?*

The most logical method, I would think, is a vim algorithm that reproduces
what I did initially manually (aurgh) using the simple vi sequence below:
1. cd directory-containing-hundreds-of-openvpn-config-files
2. vim *.ovpn
3. :103
4. ~r include.txt
5. :w
6. :n

What would work, I think, is to break the target file into two parts:
A. The first half above line 103
B. The second half below line 103
And then I could add the desired line in between those two halves:
auth-user-pass C:\\path-to\\config\\username_password.txt

In self explanatory pseudocode, that would be something like this:
a. cat/type target file #1 up to line 103 > newfile #1
b. cat/type/echo target file/line after that >> newfile #1
c. cat/type target file #1 after line 104 to EOF >> newfile #1
Repeat, rinse, lather & shave.

As I said, at the moment, much to my naive surprise, simply tacking the
desired line at the _bottom_ of the hundreds of openvpn config files
actually worked (which I simply had not expected to work at first).
for %f in (*.ovpn) do type include.txt >> %f

But the batch elegance of inserting the line in the middle of the
text file (just after line 103 or replacing line 103) just isn't there.

Note also that true elegance would be to search and replace line 103
since it could be that at some time in the future, the line numbers change.

In summary, it's a useful skill perhaps to have a trick up our sleeves to
be able to insert a line in a file using the Windows command line, so if
anyone has an already working method, I would love to test it out for the
team and then I could report back on the success of that elegant method.

[Paul]

Post by Andy Burnelli

Post by Paul
The first step in a project like this, is breaking down
the process into pseudocode, so that anyone can follow it.
If you can explain to people what you're doing, successfully,
then programming it is a doddle. If your specification is
fuzzy, your program will also be fuzzy.

Hi Paul,
(This is written in explanatory-pseudocode so others can easily follow it.)
I am familiar (albeit rather unduly rusty) with Unix shell programming
having spent most of my grad school & Silicon Valley time on the 'nix's.
And I wrote a tutorial, long ago when it first came out, for installing WSL
on Windows in five minutes, along with Cygwin tutorials in long years past.
But I'm trying to see if I can do this either with a vim macro or better
yet, in the Windows command line shell, where luckily I have a real world
example anyone can see which is easily reproduced by doing the following
(which I provide so that others can _benefit_ from zillions of free VPNs!).
A. Go to http://vpngate.net
B. In the 7th column named "OpenVPN" are the free VPN openvpn config files.
C. Press on a few of them (see why below) to download them to your system.
<https://www.vpngate.net/en/do_openvpn.aspx?fqdn=vpn371239765.opengw.net&ip=180.198.24.136&tcp=1519&udp=1381&sid=1671465724963&hid=17464287>
<https://www.vpngate.net/en/do_openvpn.aspx?fqdn=vpn371239765.opengw.net>
  filename = "vpn371239765.opengw.net (UDP 1381)"
  filename = "vpn371239765.opengw.net (TCP 1519)"
  filename = "180.198.24.136 (UDP 1381)"
  filename = "180.198.24.136 (TCP 1519)"
Up until very recently, that file is the only thing you needed to do in
order to programmatically switch VPN servers every few minutes on Windows.
Pseudocode...     set configcmd=C:\path-to\openvpn\bin\openvpn.exe
    set configdir=C:\path-to\openvpn\config\
    set configfile=(that file above chosen randomly programmatically)
    cd %configdir%
    %configcmd% %configdir%%configfile%
I could never get this batch file to run "as" admin without _being_ admin, so you need to right-click on that batch file to execute with administrator
privileges (which are needed in order to change the routing table).
Notice _all_ eight thousand of these free openvpn config files are of the
    #auth-user-pass
    auth-user-pass C:\\path-to\\config\\username_password.txt
    vpn
    vpn
Which are the (apparently) brand new requirements for adding to openvpn
    username
    password
(Note: Up until recently, this username/password requirement wasn't there!)
I have found that it works to just append (or prepend) that line, so I'm
actually already well into my workaround but the problem set remains valid.
*How do you _insert_ a line into a file using the Windows command line?*
The most logical method, I would think, is a vim algorithm that reproduces
  1. cd directory-containing-hundreds-of-openvpn-config-files
  2. vim *.ovpn
  3. :103
  4. ~r include.txt
  5. :w   6. :n
  A. The first half above line 103
  B. The second half below line 103
And then I could add the desired line in between those two halves:   auth-user-pass C:\\path-to\\config\\username_password.txt
  a. cat/type target file #1 up to line 103 > newfile #1
  b. cat/type/echo target file/line after that >> newfile #1
  c. cat/type target file #1 after line 104 to EOF >> newfile #1
Repeat, rinse, lather & shave.
As I said, at the moment, much to my naive surprise, simply tacking the
desired line at the _bottom_ of the hundreds of openvpn config files
actually worked (which I simply had not expected to work at first).
  for %f in (*.ovpn) do type include.txt >> %f
But the batch elegance of inserting the line in the middle of the text file (just after line 103 or replacing line 103) just isn't there.
Note also that true elegance would be to search and replace line 103
since it could be that at some time in the future, the line numbers change.
In summary, it's a useful skill perhaps to have a trick up our sleeves to
be able to insert a line in a file using the Windows command line, so if
anyone has an already working method, I would love to test it out for the
team and then I could report back on the success of that elegant method.

# It is not recommended to modify it unless you have a particular need.

resolv-retry infinite
nobind
persist-key
persist-tun
client
verb 3
#auth-user-pass <=== auth-user-pass login.conf

Andy Burns gets the prize on this one, with his SED command style.

sed -i '8i This is Line 8' FILE

So it would be

sed -i '103i auth-user-pass login.conf' FILE

And you simply make a file with all the FILE names
in it, and do a rectangular insert to fit lines of

sed -i '103i auth-user-pass login.conf'

in front of it. And that's your .bat file.

*******

https://gnuwin32.sourceforge.net/packages.html

https://gnuwin32.sourceforge.net/packages/sed.htm

Download the "bin" and "dep" ZIP files and unpack.

libiconv2.dll \
regex2.dll \____ drop these in "bin" folder
libintl3.dll /

bin \ <=== sed.exe is in here, add three DLL files, vpn299013327.ovpn goes here.
contrib \
man \___ unpacked directory set
manifest /
share /

cd bin
sed -i '103i auth-user-pass login.conf' vpn299013327.ovpn

******* the result, not quite... *******

resolv-retry infinite
nobind
persist-key
persist-tun
client
verb 3
auth-user-pass login.conf <=== it just did an insert ahead of the next line
#auth-user-pass Which is functionally fine. This would work.

*****************************************

This might do it. Had to escape the hash symbol.

sed -i '103s/\#auth-user-pass/auth-user-pass login.conf/' vpn299013327.ovpn

Since the string in question only appears once in the file,
you probably don't even need to tell it the line number.

But it's sed, so you could be playing with this all afternoon.

Paul

[Herbert Kleebauer]

Post by Paul
With C, there would be a temptation to write some
low level text handlers, that a textual processor
would already have. Maybe you can find a library for C
to act as an assistant, so it's not an impossibility.

Just searched for a small C compiler which doesn't need
to be installed so you can store it on an USB stick and
use it on any computer.

https://download.savannah.gnu.org/releases/tinycc/
https://download.savannah.gnu.org/releases/tinycc/tcc-0.9.27-win32-bin.zip

Just put the content of the zip file in a new folder
and all you have to do is:

tcc your_program.c

It directly creates the binary your_program.exe

And here a small C program which inserts text after
a given number of lines in a file:

________________________________________________________

/* usage: ins infile outfile */
/* after line number LINE the text TEXT is inserted */

#define LINE 5
#define TEXT "user\npass\n"

/************************************************************************
external functions used: getc(); putc(); fopen(); fclose(); exit();
************************************************************************/

#include <stdio.h>
FILE *in, *out, *fopen();

void out_string();
void print_string();
void ende();
void exit();

main(nargs,args) int nargs; char *args[];
{if (nargs!=3) ende(1);
in = fopen(args[1],"rb"); if (in==NULL) ende(2);
out = fopen(args[2],"wb"); if (out==NULL) ende(3);

int i,n=0;
while ( (i=getc(in)) != EOF)
{putc(i,out);
if (i == 10) n++;
if (n == LINE) {n++; out_string(TEXT);}
}
ende(0);
}

void out_string(p) char *p; {while (*p) putc(*p++,out);}
void print_string(p) char *p; {while (*p) putc(*p++,stdout);}

void ende(i) int i;
{static char *(meld[]) = {
/* 0 */ "no errors" ,
/* 1 */ "usage: ins infile outfile" ,
/* 2 */ "can't open input file" ,
/* 3 */ "can't open output file" ,
/* 4 */ "you never should read this" ,
/* 5 */ "you never should read this" ,
/* 6 */ "you never should read this" ,
};

print_string("\n"); print_string(meld[i]); print_string("\n");
if (in!=NULL) fclose(in);
if (out!=NULL) fclose(out);
exit(0);
}

[Chris]

Post by Andy Burnelli
But how can I _insert+ that one-line include.txt file at line 103?
How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?

What about writing a few-line C program? Or one of a language you prefer.

This is the way. Very easy to do for someone with multiple college
deg^H^H^H books.

[Andy Burns]

Post by Andy Burnelli
How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?

https://www.gnu.org/software/sed/manual/sed.html

sed -i '22i This is your new line 22' filename.txt

[Kerr-Mudd, John]

On Mon, 19 Dec 2022 07:43:43 +0000

Post by Andy Burnelli
How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?
It seems that vpngate.net vpn configuration files now require a username
(of vpn) and a password (of vpn), which isn't a big deal for most people
who don't use automation, as they'd just type the login/password in when
asked by their openvpn client graphical user interface.
But I change the Time Zone & VPN server every few minutes via batch files.
It's easy to _manually (aurgh!) add at line 103 the "password.txt" line,
but what might you think is an easy way to add that line to _every_ file?
1. cd directory-containing-hundreds-of-openvpn-config-files
2. vim *.ovpn
3. :103
4. ~r include.txt
5. :write (abbreviated as :w)
6. :next (abbreviated as :n)
1. That manual process starts in a directory of many files
2. and then edits, one by one, each of a few hundred *.ovpn files;
3. and then it jumps to line 103 (it's always in the same spot);
auth-user-pass C:\\path\\config\\passwd.txt
5. and then I write the *.ovpn file;
6. and then I skip to the next *.ovpn file.
vpn
vpn
The first being the username & the latter being the password as described
in the <http://vpngate.net> home page (this is a brand new requirement).
If I were to simply append the line to the _end_ of the *.ovpn files,
instead of inserting thatline at line 103, I could use a Windows command
for %f in (*.ovpn) do type include.txt >> %f
But how can I _insert+ that one-line include.txt file at line 103?
How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?

Back when I did a bit more than dabble with computers (late 80's) I wrote a
program to do it, with parameters 'Insert "New Line" After "Known Line"'
but in asm, so non-portable.

[Big Al]

Post by Kerr-Mudd, John
On Mon, 19 Dec 2022 07:43:43 +0000

Post by Andy Burnelli
How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?
It seems that vpngate.net vpn configuration files now require a username
(of vpn) and a password (of vpn), which isn't a big deal for most people
who don't use automation, as they'd just type the login/password in when
asked by their openvpn client graphical user interface.
But I change the Time Zone & VPN server every few minutes via batch files.
It's easy to _manually (aurgh!) add at line 103 the "password.txt" line,
but what might you think is an easy way to add that line to _every_ file?
1. cd directory-containing-hundreds-of-openvpn-config-files
2. vim *.ovpn
3. :103
4. ~r include.txt
5. :write (abbreviated as :w)
6. :next (abbreviated as :n)
1. That manual process starts in a directory of many files
2. and then edits, one by one, each of a few hundred *.ovpn files;
3. and then it jumps to line 103 (it's always in the same spot);
auth-user-pass C:\\path\\config\\passwd.txt
5. and then I write the *.ovpn file;
6. and then I skip to the next *.ovpn file.
vpn
vpn
The first being the username & the latter being the password as described
in the <http://vpngate.net> home page (this is a brand new requirement).
If I were to simply append the line to the _end_ of the *.ovpn files,
instead of inserting thatline at line 103, I could use a Windows command
for %f in (*.ovpn) do type include.txt >> %f
But how can I _insert+ that one-line include.txt file at line 103?
How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?

Back when I did a bit more than dabble with computers (late 80's) I wrote a
program to do it, with parameters 'Insert "New Line" After "Known Line"'
but in asm, so non-portable.

I'm running Linux, but Geany is a text editor that will do search and replace on all files loaded, kinda like a batch edit.
If the line 102 was the same, you could search and replace line102 with line102\nline103.
Not sure how or what editor in Windows you could use to batch edit files, or how you would get a newline character to be
accepted as part of the search & replace action.
Al

[Herbert Kleebauer]

Post by Andy Burnelli
How can one most easily insert contents of a file (or a line of text)
at a given line-number location inside each of hundreds of text files?
If I were to simply append the line to the _end_ of the *.ovpn files,
instead of inserting thatline at line 103, I could use a Windows command
for %f in (*.ovpn) do type include.txt >> %f
But how can I _insert+ that one-line include.txt file at line 103?

If there are no poisoned characters (^%!) in the file you can use:

@echo off
setlocal enabledelayedexpansion
if exist out.txt del out.txt

set /a n=0

for /f "tokens=1,* delims=]" %%i in ('find /n /v "" ^< "in.txt"') do (
echo.%%j>>out.txt
set /a n=n+1
if !n!==103 echo vpn_user>>out.txt
if !n!==103 echo vpn_pass>>out.txt)

[R.Wieser]

Andy,

Post by Andy Burnelli
But how can I _insert+ that one-line include.txt file at line 103?

Use the same NT batch environment to read a file line-by-line and than count
its lines.

I'm sure Google will offer you all the needed help to how to do such
line-by-line reading and counting - though you might have to split that up
and search for them seperatily.

Regards,
Rudy Wieser