If you happen to see that your file has gotten code page 1252 aka Windows-Latin 1 in it, then you’ll have a variety of random characters like ^M or ?~@~Y or ?~@~\\ or ?~@~] .<\/p>\n
Well, I wrote a script that removes these guys and makes sure that the file format of Unix is respected. Here it is:<\/p>\n
\n#!\/bin\/bash
\n#
\n# By: barce[a t]codebelay.com
\n# ——————-
\n# this script replaces microsoft special chars with plain ol’ ascii
\n#
\n# usage: .\/repl.bash filename
\n#<\/p>\n
# replace ^M characters
\nperl -pi -e ‘s\/\\x{0D}\\x{0A}\/\\x{0A}\/g’ $1<\/p>\n
# replace garbage with single-quotes
\n# ?~@~Y
\nperl -pi -e ‘s\/\\x{E2}\\x{80}\\x{99}\/\\x{27}\/g’ $1
\nperl -pi -e ‘s\/\\x{80}\\x{99}\/\\x{27}\/g’ $1
\nperl -pi -e ‘s\/\\x{80}\\x{9c}\/\\x{27}\/g’ $1
\nperl -pi -e ‘s\/\\x{80}\\x{9d}\/\\x{27}\/g’ $1<\/p>\n
# replace garbage with asterisk
\n# ?~@?
\n# e280 a2
\nperl -pi -e ‘s\/\\x{E2}\\x{80}\\x{A2}\/\\x{2A}\/g’ $1<\/p>\n
# replace garbage quotes with plain quotes
\n# start: ?~@~\\
\n# close: ?~@~]
\n# e2 809c
\nperl -pi -e ‘s\/\\x{E2}\\x{80}\\x{9C}\/\\x{22}\/g’ $1
\nperl -pi -e ‘s\/\\x{E2}\\x{80}\\x{9D}\/\\x{22}\/g’ $1<\/p>\n
# replace garbage hyphens with plain hyphens
\nperl -pi -e ‘s\/\\x{E2}\\x{80}\\x{93}\/\\x{2D}\/g’ $1<\/p>\n
# replace garbage with ellipsis
\nperl -pi -e ‘s\/\\x{E2}\\x{80}\\x{A6}\/\\x{2E}\\x{2E}\\x{2E}\/g’ $1\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"
Hey folks, this is a goody but quicky. First off, respect the character encoding of a file. I don’t know how many devs out there violate this rule, but if you’re like me and Joel On Software, you’ll agree that you should respect the character encoding of a file. If you happen to see that […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[170,4],"tags":[210,208,211,213,214,207,209,212],"class_list":["post-320","post","type-post","status-publish","format-standard","hentry","category-command-line","category-how-to","tag-ascii","tag-bash","tag-code-page-1252","tag-latin-1","tag-perl","tag-shell-scripting","tag-utf-8","tag-windows-latin-1"],"_links":{"self":[{"href":"https:\/\/www.codebelay.com\/blog\/wp-json\/wp\/v2\/posts\/320","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.codebelay.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.codebelay.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.codebelay.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.codebelay.com\/blog\/wp-json\/wp\/v2\/comments?post=320"}],"version-history":[{"count":0,"href":"https:\/\/www.codebelay.com\/blog\/wp-json\/wp\/v2\/posts\/320\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.codebelay.com\/blog\/wp-json\/wp\/v2\/media?parent=320"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.codebelay.com\/blog\/wp-json\/wp\/v2\/categories?post=320"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.codebelay.com\/blog\/wp-json\/wp\/v2\/tags?post=320"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}