Linux One Liner – Parsing long HTML urls

Ever wanted to look at a long HTML URL more easily, say to investigate a parameter. Here is a search from MapQuest.

http://www.mapquest.com/maps/map.adp?formtype=address&addtohistory=&address=10%20Market%20St&city=San%20Francisco&state=CA&zipcode=94111%2d4801&country=US&geodiff=1

<br /> $ echo "[insert url here]" | | tr "&#038;?" "n"<br />

This produced for the above URL the following output.
`

http://www.mapquest.com/maps/map.adp

formtype=address
addtohistory=
address=10%20Market%20St
city=San%20Francisco
state=CA
zipcode=94111%2d4801
country=US
geodiff=1
`

The Translate command tr does however replace both the & and ? characters. There are of course many more approaches like.

<br /> echo "[insert url here]" | sed -e "s/&#038;/\n/g" -e "s/?/\n/g"<br />

You can easily preserve the & and ? characters extending the syntax with
<br /> echo "[insert url here]" | sed -e "s/&#038;/\n&#038;/g" -e "s/?/\n?/g<br />

This produces.
`

http://www.mapquest.com/maps/map.adp

?formtype=address
&addtohistory=
&address=10%20Market%20St
&city=San%20Francisco
&state=CA
&zipcode=94111%2d4801
&country=US
&geodiff=1
`

Now don’t get me started with the awk command. One of my popular books is Sed & Awk. If you do any detailed Shell scripting, this is a very handy guide.

Tagged with: General Linux One Liners

Related Posts

More CPUs or Newer CPUs

In a CPU-bound database workload, regardless of price, would you scale-up or scale-new? What if price was the driving factor, would you scale-up or scale-new? I am using as a baseline the first available AWS Graviton2 processor for RDS (r6g).

Read more

An Interesting Artifact with AWS RDS Aurora Storage

As part of using public datasets with my own Benchmarking Suite I wanted upsize a dataset for larger volume testing. I have always used the INFORMATION_SCHEMA.TABLES data_length and index_length columns as a sufficiently accurate measurement for actual disk space used.

Read more

How long does it take the ReadySet cache to warm up?

During my setup of benchmarking I run a quick test-sysbench script to ensure my configuration is right before running an hour+ duration test. When pointing to a Readyset cache where I have cached the 5 queries used in the sysbench test, but I have not run any execution of the SQL, throughput went up 10x in 5 seconds.

Read more