67

I spent some time manually editing a post replacing Mathematica ASCII \[Alpha] with Unicode α. I did this by laboriously choosing Copy as LaTeX, pasting into the edit box, and then copying the Unicode symbol from the preview below. This made me realize I am lacking a "Copy as Unicode string" function in Mathematica.

How can I most easily copy an expression such as:

Mathematica graphics

In Unicode:

αβ + Mod[δΨ, 2 ⁢ρ^2]
Alexey Popkov
  • 61,809
  • 7
  • 149
  • 368
Mr.Wizard
  • 271,378
  • 34
  • 587
  • 1,371

10 Answers10

30

Since a native method is not forthcoming, I shall post my file based circumvention, for Windows.

You will need to have this utility in the command path (it apparently is stock with Windows 7).

copyUnicode[expr_] := Run["clip <",
   Export["$Clipboard.temp", ToString[expr, InputForm],
          "Text", CharacterEncoding -> "Unicode"] ];

Usage:

expr = \[Alpha]\[Beta] + Mod[\[Delta]\[CapitalPsi], 2\[InvisibleTimes]\[Rho]^2];

copyUnicode[expr]

This leaves the following text in the Windows Clipboard:

αβ + Mod[δΨ, 2*ρ^2]

Here is a version of the function that holds (does not evaluate) the expression:

SetAttributes[copyUnicode, HoldFirst]

copyUnicode[expr_, form_: InputForm] := 
  Run["clip <", 
   Export["$Clipboard.temp", ToString[Unevaluated@expr, form], "Text", 
    CharacterEncoding -> "Unicode"]];

Now:

Plot[\[Alpha], {\[Alpha], 0, 10}] // copyUnicode

Puts in the Windows Clipboard:

Plot[α, {α, 0, 10}]
Mr.Wizard
  • 271,378
  • 34
  • 587
  • 1,371
  • 3
    I think clip is native in at least win-7. Perhaps you should note that this places the output in the system clipboard. The output of your copyUnicode function is just a 0 (at least, on my PC). You have to do a ctrl-v paste afterwards. A MMA Paste doesn't seem to work. – Sjoerd C. de Vries Feb 02 '12 at 21:10
  • 2
    Very useful function! Why not make it available for any code sample? (Now it works only for single expression…) – xzczd Nov 23 '12 at 14:09
  • @xzczd I'll see if I can improve it. – Mr.Wizard Nov 23 '12 at 23:56
  • There's a built-in CopyToClipboard function, but it doesn't seem to work, at least on Linux. – Mechanical snail May 21 '13 at 01:46
  • Does it possible to convert string to Plot[\[Alpha], {\[Alpha], 0, 10}] or Cell[BoxData["\<\"Plot[\\\\[Alpha], {\\\\[Alpha], 0, 10}]\"\>"], "Output"}] to Unicode Form? Because I use one function something like InputCell2SE@cellExpression//CopyToClipboard copy InputCell to SE which turns out Plot[\[Alpha], {\[Alpha], 0, 10}], and I used FrontEnd..."InputText" – HyperGroups Jun 19 '13 at 15:08
  • @HyperGroups I added a variation of the function that holds the expression. – Mr.Wizard Jun 19 '13 at 15:30
  • A native method may be forthcoming: see this meta discussion. – shrx Jun 19 '13 at 17:05
  • @Mr.Wizard Maybe Leonid will come up with an OS independent solution, but if not, we may want to combine our approaches. I can probably simplify my approach to use pbcopy (similar to clip, I think). But I would say that a more modular design would be to let copyUnicode only accept String in the first place, and leave it to the calling function to provide the string. I.e., I wouldn't need to use HoldFirst on copyUnicode itself then. Anyway, it depends on how it's going to be used. Should I try to write a combined version of our Mac/PC solutions? – Jens Jun 19 '13 at 19:36
  • Hi, I post one question to describe my usage, need your attention. http://mathematica.stackexchange.com/questions/27289/convert-alpha-to-unicode-form – HyperGroups Jun 20 '13 at 01:46
  • 1
    A more modern version that does not use a temporary file: RunProcess["clip", All, ExportString["αβ + Mod[δΨ, 2 ⁢ρ^2]", "Text", CharacterEncoding -> "Unicode"]] – Stephen Powell Oct 07 '21 at 09:44
24

Declaration:

This method for Windows is based on the .NET code from Todd Gayley's this wonderful answer. My .NET knowledge is absolutely ZERO, all credit goes to Todd.

Code:

The main idea is to extract the "Input"-style code string, convert it to the UTF-16 little endian form, which is the standard byte order in Windows, feed the bytes to system clipboard by using .NET API.

The main functions are placed in a package:

Needs["NETLink`"]
InstallNET[];
BeginPackage["UniCodeCopy`"]

mmaUnicodeToUTF16LE::usage = ToString[Row[{Style["mmaUnicodeToUTF16LE[_Integer]", Bold], " convert integer list obtained from ", Style[Row[{"ToCharacterCode[", Style["string", Italic], ", &quot;Unicode&quot;]"}], Bold], " to integer list consistent with the UTF‐16 Little Endian standard."}], StandardForm]; stringToUTF16LE::usage = ToString[Row[{Style["stringToUTF16LE[_String]", Bold], " convert ", Style["string", Italic], " to integer list consistent with the UTF‐16 Little Endian standard."}], StandardForm]; WriteToClipboardUnicode::usage = ToString[Row[{Style["WriteToClipboardUnicode[_String]", Bold], " write ", Style["string", Italic], " to Windows clipboard using the UTF‐16 Little Endian encoding."}], StandardForm];

Begin["Private"]

Clear[mmaUnicodeToUTF16LE] mmaUnicodeToUTF16LE[bytecode_Integer] := PadRight[#, 2 Ceiling[Length@#/2]] &@ Join[Most[#] - Rest[# 2^8], #[[{-1}]]] &@ DeleteCases[FixedPointList[BitShiftRight[#, 8] &, bytecode], 0]

Clear[stringToUTF16LE] stringToUTF16LE[str_String] := mmaUnicodeToUTF16LE /@ ToCharacterCode[str, "Unicode"] // Flatten // Join[#, {0, 0}] &

Clear[WriteToClipboardUnicode] WriteToClipboardUnicode[str_String] := Module[{bytecode, strm, dataObject}, bytecode = stringToUTF16LE[str]; NETLinkNETBlock[ strm = NETLinkNETNew["System.IO.MemoryStream", bytecode]; dataObject = NETLinkNETNew[&quot;System.Windows.Forms.DataObject&quot;]; dataObject@NETLinkSetData["Text", strm]; NETLinkLoadNETType[&quot;System.Windows.Forms.Clipboard&quot;]; SystemWindowsFormsClipboard`SetDataObject[dataObject] ] ]

End[] EndPackage[]

Then execute the following code, which will generate a palette with a UniCode Copy button:

Button["UniCode Copy",
    Module[{codestr},
        timestamp = {};
        FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]];
        codestr = 
            NotebookGet[ClipboardNotebook[]][[1, 1, 1]] // 
                StringReplace[#, {"\\\n" -> "", 
                            "\n" ~~ space : " " ... :> 
                                StringJoin["\n", ConstantArray[space, 4]]}] &;
        UniCodeCopy`WriteToClipboardUnicode[codestr]
        ],
    Method -> "Queued"
    ]

SelectionMove[EvaluationNotebook[], Previous, Cell]; FrontEndExecute[FrontEndToken["GeneratePalette"]]; FrontEndExecute[FrontEndToken["Clear"]];

To use it, select the Cells you want to copy as input text code, then press the button. The package can be put in a file and be auto-loaded when MMA start, the palette can be installed to system menu.

Examples:

Unicode expressions in Mathematica Notebook:

test unicode expressions

Text generated directly by the above code:

"αβγδϵζηθχϕϡΔΥϖϒϠíãéõÂÆŁØÝ∟△♆♇√÷∇■○▫✶✓¥¶†︵│︷中文汉字日文コンピュータ"

α β + Mod[δΨ, 2 ρ^2]

Silvia
  • 27,556
  • 3
  • 84
  • 164
  • Very nice. Thanks! – Mr.Wizard Apr 01 '14 at 22:10
  • @Mr.Wizard It's my honour :) Needed this tool myself for a loong time. And there are still problems, such as some characters from private area get copied, e.g. a. Maybe a whitelist is needed. – Silvia Apr 01 '14 at 22:16
  • Arguably copy of characters such as \[Transpose] as  is acceptable as they can still be pasted into a Notebook, although it would be nice to also have the option to copy the expanded FullForm of such private letters. – Mr.Wizard Apr 02 '14 at 02:23
  • @Mr.Wizard One possible way came into my mind: Read the Boxes, identify the private characters (especially those used as functions like \[Transpose]), convert the corresponding parts into InputForm(how?), then make the copy. – Silvia Apr 02 '14 at 20:41
  • @Mr.Wizard Wow thank you for your bounty! :D – Silvia Apr 07 '14 at 23:59
  • I think you earned it. Apparently the foundation was presented by Todd Gayley but I never built upon it. A button like this is an actual solution, unlike having to wrap code blocks in copyUnicode (with or without a Hold attribute) as I have been doing. If it were platform and version independent I would have Accepted it, but I did not so as to leave room for future additions. – Mr.Wizard Apr 08 '14 at 01:03
  • Thanks. I believe a similar method can be realized with JLink, that will be platform independent. But again, I have zero knowledge on Java.. And why not build a button with your copyUnicode? – Silvia Apr 08 '14 at 01:08
  • I did that and it sorta worked, but I had trouble making it robust. So far this method hasn't failed. Anyway I like this method because it brings an entirely different approach that doesn't rely on exporting to a file, etc. Thanks again for sharing it. – Mr.Wizard Apr 08 '14 at 06:06
  • @Mr.Wizard You're welcome! I'm using it daily now :) I updated the code into a ready-to-use package, hope it makes thing easier. – Silvia Apr 08 '14 at 07:05
  • Why so cool you! – LCFactorization Feb 02 '16 at 06:13
  • I found \[Rule] cannot be converted into unicode. Does that mean there is no such a corresponding symbol there? – LCFactorization Feb 14 '16 at 09:11
  • 2
    @LCFactorization I think the \[Rule] symbol in MMA, whose Unicode is U+F522, is in the Basic Multilingual Plane (U+E000–U+F8FF), one of the Private Use Area in Unicode standard, thus may not be printable in 3rd party software. – Silvia Feb 14 '16 at 16:03
  • thank you! It seems for this symbol, -> is a better way to represent. @xzczd suggests an online versiion converter, which is also very useful and deserves a try: http://steampiano.net/msc/ BTW, are you interested in solving tough nonlinear PDE system problems by NDSolve or similar solvers? – LCFactorization Feb 15 '16 at 10:03
  • 1
    @LCFactorization xzczd's web service looks very nice, thanks for sharing! Regarding the PDE, if you have any examples, please feel free to ping me in chat or email me :) – Silvia Feb 18 '16 at 14:20
15

Edit 2: A new version of the Mac solution with button is listed below

Fixed problem with pasting into textarea

In some applications on Mac, copying as Unicode from Mathematica already works without having to do any postprocessing. However, it doesn't work in textarea fields in web browsers.

Nevertheless, if you're willing to do a few additional mouse clicks, the Unicode forms can be brought into the browser by taking the detour through one of those applications that do support Mathematica's characters. One convenient choice is to open a new window in TextEdit and paste your original Mathematica code into it first. In the default RichText mode, TextEdit displays the special characters as Unicode glyphs. Then you can simply copy whatever you just pasted back from TextEdit and paste it into the browser. It seems that as soon as TextEdit is recognized as the provider of the Unicode text on the pasteboard, there is no conversion back to the Mathematica representation, so you get the correct appearance in the browser.

In the following, I use the same idea in order to automate the conversion. Instead of TextEdit, I go through a temporary file in RichText (RTF) format on the provider side. The Mac pasteboard does support RTF as a format for data on the pasteboard, but this isn't accepted by the browsers I've recently tried when attempting to paste into textarea.

Therefore, I have to add one additional step: using the textutil tool which is built into OS X, I convert the RTF file to a regular text file with Unicode characters first. Then I read the text file back in and put it on the pasteboard.

Of course, this means it's only going to work on Mac OS X because it uses Cocoa bindings in the built-in Python interpreter:

copyAsUnicode[t_] := 
 Module[{out = 
    FileNameJoin[{$TemporaryDirectory, 
      "MathematicaOutput" <> StringJoin[Map[ToString, DateList[]]]}]},
   Export[out <> ".rtf", t];
  Run["textutil -convert txt " <> out <> ".rtf -output " <> out <> 
    ".txt"]; 
  Run["printf \"from AppKit import *\n\
board=NSPasteboard.generalPasteboard()\n\
content=NSData.dataWithContentsOfFile_('" <> out <> 
    ".txt')\nboard.declareTypes_owner_([NSStringPboardType], None)\n\
board.setData_forType_(content, NSStringPboardType)\n\
\" | /usr/bin/python"];
  DeleteFile[{out <> ".txt", out <> ".rtf"}]]

The idea is to export to RTF and read the result to the clipboard outside of Mathematica. The function is invoked for example as copyAsUnicode["αβ+Mod[δΨ+ρ2]"]. This example itself was copied that way, too, i.e., I typed copyAsUnicode["copyAsUnicode[\"αβ+Mod[δΨ+ρ2]\"]"], which I again copied the same way... OK, I think you get the idea.

Of course the next step would be to make this into a Palette that acts on the NotebookSelection, but the above is the main step. Maybe someone else knows how to do something like this in other operating systems (I don't).

Installing this function as a button:

Responding to the comment, here I'm just listing the same function as above, but wrapped in Silvia's code to make it into a button. It requires no package loading because I inlined everything into the button code:

Button["UniCode Copy", Module[{codestr},
FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]];
codestr = NotebookGet[ClipboardNotebook[]][[1, 1, 1]];
 Module[{out = 
    FileNameJoin[{$TemporaryDirectory, 
      "MathematicaOutput" <> StringJoin[Map[ToString, DateList[]]]}]},
   Export[out <> ".rtf", codestr];
  Run["textutil -convert txt " <> out <> ".rtf -output " <> out <> 
    ".txt"]; 
  Run["printf \"from AppKit import *\n\
board=NSPasteboard.generalPasteboard()\n\
content=NSData.dataWithContentsOfFile_('" <> out <> 
    ".txt')\nboard.declareTypes_owner_([NSStringPboardType], None)\n\
board.setData_forType_(content, NSStringPboardType)\n\
\" | /usr/bin/python"];
  DeleteFile[{out <> ".txt", out <> ".rtf"}]]],
Method -> "Queued"]
SelectionMove[EvaluationNotebook[], Previous, Cell];
FrontEndExecute[FrontEndToken["GeneratePalette"]];
FrontEndExecute[FrontEndToken["Clear"]];

Now you can also install this permanently by going to Palettes > Install Palette and selecting a name for the palette.

Jens
  • 97,245
  • 7
  • 213
  • 499
  • This looks good, but since I cannot test it on Windows I will leave it to Mac users to upvote. Thank you for your answer. – Mr.Wizard Feb 02 '12 at 04:27
  • Hi, I posted one question with little difference from directly copy Input Expression to SE and need your attention. mathematica.stackexchange.com/questions/27289/… – HyperGroups Jun 20 '13 at 01:47
  • @Jens, this is a nice workaround (and it still works). Any idea if there are plans for this to be integrated into a Palette as you suggested? – Rico Picone May 13 '14 at 18:30
  • @RicoPicone Thanks, I added the button code, hopefully that should put it on equal footing with the windows solution (except for potential shortcomings of the Mac Unicode handling, which I have no control over...). – Jens May 13 '14 at 19:33
  • @Jens, thanks for your work on this. I can't get it to work. I get the message (copy from clipboard option) that the clipboard contents are invalid. I'll try to save it as a file, next. – Rico Picone May 14 '14 at 14:32
  • Is it working if you use the copyAsUnicode function by itself? I'll get back to this later, but any additional info would help. Is the error message generated when you try to paste into some external application (which one)? I'm trying to understand if it's the button code or the original function that isn't working (it works for me). – Jens May 14 '14 at 15:17
  • On my Mac this answer works fine (including the copy button) in Mathematica versions 8 and 10. – Jens Sep 20 '14 at 22:13
  • For me both the button and the function work, on MMA 10, OSX 10.9.5. One thing is that an additional space seems to be added at the front of the string, which is only a minor inconvenience. (+1) – Jacob Akkerboom Sep 30 '14 at 13:12
  • On newer Macs, the python executable no longer exists at the location I used in the code above. You'll have to first install a version of python and pyobjc so that AppKit can be found. Then change the path in my code from /usr/bin/python to the appropriate location. – Jens Apr 19 '23 at 15:47
15

Here's how to add a permanent menu item and keyboard shortcut for Silvia's solution. (This can be easily adapted to Jens' solution for Mac OS.)

enter image description here

1. Add Package

Copy Silvia's first code block that starts with Needs["NETLink`"] to a file and save it as UniCodeCopy.m in one of the directories included in $Path.

2. Initialize the Front End

Save the following to the file $UserBaseDirectory/Autoload/FrontEnd/init.m (create it if needed):

Needs["UniCodeCopy`"]

CopySelectionAsUnicode[] :=
    Module[{selection, code},
           FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]];
           selection = NotebookGet[ClipboardNotebook[]][[1, 1, 1]];
           code = StringReplace[selection, {"\\\n" -> "",
                                            "\n" ~~ space : " " ... :> StringJoin["\n", ConstantArray[space, 4]]}];
           UniCodeCopy`WriteToClipboardUnicode[code];]

FrontEndExecute[
    FrontEnd`AddMenuCommands["Copy",
                             {MenuItem["Copy as Unicode",
                                       FrontEndExecute[CopySelectionAsUnicode[]],
                                       (* Alt + U *)
                                       MenuKey["u", Modifiers -> {"Command"}],
                                       System`MenuEvaluator -> Automatic,
                                       Method -> "Queued"]}]]

For key bindings, the possible modifiers are "Shift", "Control", "Command" (Alt) and "Option" (Alt).

(It took me a few hours to figure this out. I hope this proves useful for other struggling novices.)

ens
  • 591
  • 3
  • 8
  • 2
    Very nice! Thanks for sharing :) – Silvia Aug 10 '14 at 06:46
  • 1
    Are you saying that $UserBaseDirectory/Autoload/FrontEnd/init.m should be created, because there is no such file on my Windows 8.1 computer? There is, however, the file $UserBaseDirectory/FrontEnd/init.m on my computer, but appending your code to the end of that file caused Mathematica 10.0.2.0 to crash during initialization. Thanks in advance for your assistance. – bbgodfrey Mar 08 '15 at 22:40
  • Yes, just create it. I've added that to my post. – ens Mar 08 '15 at 22:53
  • It works well. Thank you. – tanghe2014 Jun 01 '20 at 01:08
13

Here is a function that copies a Unicode string to the clipboard using JLink:

Needs["JLink`"];
InstallJava[];
LoadJavaClass["java.awt.Toolkit", AllowShortContext -> False];

uniclip[s_String] := JavaBlock[ javaawtToolkit`getDefaultToolkit[]@getSystemClipboard[]@setContents[#, #]& @ JavaNew["java.awt.datatransfer.StringSelection", s] ];

From Mathematica 12.3 onward, we can use ExternalFunction instead of JLink (thanks to Ben Izd for pointing this out):

uniclip = ExternalFunction["Java",
  "static void uniclip(String string) {
     var s = new java.awt.datatransfer.StringSelection(string);
     java.awt.Toolkit.getDefaultToolkit().getSystemClipboard().setContents(s, s);
   }"
];

Example usage:

expr = \[Alpha]\[Beta] + Mod[\[Delta]\[CapitalPsi], 2\[InvisibleTimes]\[Rho]^2];

uniclip @ ToString[expr, InputForm]

After evaluating these expressions, the clipboard contains αβ + Mod[δΨ, 2*ρ^2].

WReach
  • 68,832
  • 4
  • 164
  • 269
  • 1
    This is great! I suppose it should work on all platforms? – Silvia Sep 07 '14 at 20:43
  • Great - I see no problems on Mac OS X 10.7.5 with Mathematica 10. – Jens Sep 07 '14 at 21:33
  • @Silvia I can only vouch for it working on Windows, but by rights it should work on all platforms. – WReach Sep 07 '14 at 22:20
  • 3
    I believe this is the first multiplatform solution. Provide complete self-contained package code to add this as a menu item and shortcut key (in the context menu if you can manage it), and I'll Accept it so long as no failures appear. Thanks! – Mr.Wizard Sep 08 '14 at 06:55
  • @WReach Are you disinterested in this or do you simply have other priorities? – Mr.Wizard Sep 20 '14 at 11:01
  • 2
    @Mr.Wizard I spent a couple of hours today trying to fulfill the request, but failed. I kept hitting race conditions in re-entrant front-end calls while trying to hook up the context menu. This is because the only way I know how recover the text selection involves a front-end call, and the menu item invocation is already a front-end call. I can't really spend more time on it right now, so if anyone wants to incorporate uniclip into another answer that adds menu items and shortcut keys, feel free. – WReach Sep 20 '14 at 18:33
  • Thanks @WReach ! I appreciate the effort. :-) – Mr.Wizard Sep 20 '14 at 20:51
  • 1
    It seems uniclip is broken on Windows 10 in recently released M13.2. One of the error messages was "Java failed to load class sun.awt.windows.WClipboard." – Silvia Dec 22 '22 at 08:26
  • 1
    @Silvia It seems in 13.2 java version is 17 and there is some module access issue which I think is related to JDK 16 feature (being strongly encapsulated by default). Since 12.3 we have "Java" for ExternalEvaluate which can be used as a workaround for the error. We can use this definition of uniclip which is exactly the same but in Java: – Ben Izd Dec 22 '22 at 17:00
  • 2
    uniclip = ExternalEvaluate["Java", "import java.awt.Toolkit; import java.awt.datatransfer.StringSelection; import java.awt.datatransfer.Clipboard; public static void copy(String string){ StringSelection selection = new StringSelection(string); Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard(); clipboard.setContents(selection, selection); } "] – Ben Izd Dec 22 '22 at 17:00
  • @BenIzd Thanks for pointing out that option... I have incorporated a variant into my response. – WReach Dec 22 '22 at 18:03
  • @BenIzd @WReach Thanks for the prompt solution! And thanks for pointing out ExternalEvaluate. – Silvia Dec 22 '22 at 18:44
12

I use a small web application for when there are too many to convert by hand.

cormullion
  • 24,243
  • 4
  • 64
  • 133
11

Here's a version that doesn't require a temporary file.

Linux (needs xclip)

SetAttributes[copyUnicode, HoldAll];
copyUnicode[expr_] := With[{
      stream = OpenWrite["!xclip -in -selection clipboard", CharacterEncoding -> "UTF-8"]
   },
   WriteString[stream, ToString[Unevaluated@expr, InputForm]];
   Close@stream;
];

Example: executing the cell

Cell[BoxData[
 RowBox[{"copyUnicode", "[", 
  RowBox[{
   RowBox[{
    RowBox[{
     SuperscriptBox["x", "2"], "\[SmallCircle]", 
     RowBox[{"{", 
      RowBox[{
       RowBox[{"\[LeftFloor]", "\[Alpha]", "\[RightFloor]"}], ",", 
       "\"\<\[LeftFloor]\[Alpha]\[RightFloor]\>\""}], "}"}]}], 
    "\[PlusMinus]", 
    RowBox[{
     SqrtBox["5"], "\[CirclePlus]", "\[HappySmiley]"}]}], 
   "<=", 
   "\"\<\[Integral]\[PartialD]\[RightArrow]\[Union]\[Sum]\[Infinity]\
\[Element]\>\""}], "]"}]], "Input"]

gives x^2 ∘ {Floor[α], "⌊α⌋"} ± Sqrt[5] ⊕ ☺ <= "∫∂→⋃∑∞∈".

Windows

Not tested, but it should work if you use "!clip"instead of"!xclip -in -selection clipboard"`. You might have to change the encoding to UTF-16.

Caveats

  • Note that certain characters get ASCIIfied anyway in InputForm when not inside a string.
  • Mathematica uses non-standard private-use code points for some characters like U+211D , even when a standard code point exists, so the output will be wrong if the input contains such characters.
Mechanical snail
  • 2,422
  • 1
  • 16
  • 35
  • Your caveats also apply to Mac OS X, so there's more work to be done before it becomes 100% reliable... (+1) – Jens Jun 19 '13 at 20:21
  • Another caveat to add: some special characters, such as \[UpEquilibrium], get converted incorrectly or not at all, no matter what I try. So I don't think it's possible to get a perfect translation to Unicode in general. – Jens Jun 19 '13 at 20:46
4

Here's a method that's really no different from the standard ones except it's generalized for copying cells and, theoretically, for Windows and Unix.

We'll use RunProcess because this exists now, and tweak the ProcessEnvironment to make it work for Mac:

FrontEnd`unicodeCopy[s_String,
   enc : _String | Automatic : Automatic,
   lang : _String | Automatic : Automatic
   ] :=
  With[{
    encoding = Replace[enc, Automatic :> $CharacterEncoding],
    language = Replace[lang, Automatic :> $Language]
    },
   If[#["ExitCode"] =!= 0,
      FrontEnd`unicodeCopy::copyerr = "error in copy process:\n ``";
      Message[FrontEnd`unicodeCopy::copyerr, #["StandardError"]];,
      #["StandardOuput"];
      ] &@
    RunProcess[
     Switch[$OperatingSystem,
      "Windows",
      "clip",
      "MacOSX",
      "pbcopy",
      "Unix",
      {"xsel", "-b"}
      ],
     All,
     ExportString[s,
      "String",
      CharacterEncoding -> encoding
      ],
     ProcessEnvironment -> <|
       "LANG" ->
        Replace[ToLowerCase@language,
          "english" -> "en_US"
          ] <> "." <>
         Replace[encoding, {
           "UTF8" -> "UTF-8",
           "Unicode" -> "UTF-16"
           }],
       "PATH" -> Environment["PATH"]
       |>
     ]
   ];

(I'm using FrontEnd` for when I dump to "MenuSetup.tr", just so it looks consistent)

Now we'll define a bunch of junk to generalize to make it work across an NB:

$unicodeTR =
  FrontEnd`FindFileOnPath["UnicodeCharacters.tr", 
    "PrivatePathsTextResources"] // FrontEndExecute;
$unicodeReplacements = #[[2]] -> 
     ToExpression[
      "\"" <> StringReplace[#[[1]], {"x" -> ":", 
         StartOfString ~~ "0" -> "\\"}] <> "\""] & /@
   Rest[
    StringSplit@StringSplit[Import[$unicodeTR, "String"], "\n"]
    ];

FrontEnd`UnicodeCopy[c : {__Cell}] :=
  Replace[
   MathLink`CallFrontEnd[
    ExportPacket[Cell[CellGroupData[c]], "InputText"]
    ], {
    {s_String, __} :>
     FrontEnd`unicodeCopy@
      StringReplace[s,$unicodeReplacements]
    }];
FrontEnd`UnicodeCopy[c_Cell] :=
  Replace[
   MathLink`CallFrontEnd[
    ExportPacket[c, "InputText"]
    ], {
    {s_String, __} :>
     FrontEnd`unicodeCopy@
      StringReplace[s,$unicodeReplacements]
    }];
FrontEnd`UnicodeCopy[b_BoxData | _TextData] :=

  FrontEnd`UnicodeCopy@Cell[b];
FrontEnd`UnicodeCopy[b_RowBox] :=

  FrontEnd`UnicodeCopy@Cell[BoxData@b];
FrontEnd`UnicodeCopy[
   nb : _NotebookObject | _FrontEnd`NotebookObject | \
_FrontEnd`SelectedNotebook | _FrontEnd`InputNotebook | \
_FrontEnd`ButtonNotebook | _FrontEnd`EvaluationNotebook | \
_FrontEnd`MessagesNotebook | _FrontEnd`HelpBrowserNotebook | \
_FrontEnd`ClipboardNotebook
   ] :=
  FrontEnd`UnicodeCopy[Evaluate@NotebookRead@nb];
FrontEnd`UnicodeCopy[
   nb : _InputNotebook | _EvaluationNotebook | _ClipboardNotebook | \
_ButtonNotebook | _MessagesNotebook | _HelpBrowserNotebook] :=

  FrontEnd`UnicodeCopy@Evaluate@nb;
FrontEnd`UnicodeCopy[Optional[Automatic, Automatic]] :=

  FrontEnd`UnicodeCopy@InputNotebook[];
FrontEnd`UnicodeCopy[s_String] :=
  FrontEnd`unicodeCopy[s];
FrontEnd`UnicodeCopy[e_] :=

  FrontEnd`UnicodeCopy[Evaluate@ToString[Unevaluated[e], InputForm]];
FrontEnd`UnicodeCopy~SetAttributes~HoldFirst

Then I'll add this to my MenuSetup using the framework I wrote up here and whose up-to-date implementation lives here

FEMenuSetupAdd[
 {"Edit", "Copy As", 5},
 "Unicode" -> KernelExecute[ToExpression["FrontEnd`UnicodeCopy[]"]],
 MenuEvaluator -> Automatic,
 System`MenuKey["C", System`Modifiers -> {"Control"}]
 ]

This now lives under the "Copy As" menu:

copy as

And it can be used by function:

FrontEnd`UnicodeCopy["asdasd\[Alpha]\[Beta]\[Gamma]"]

asdasdαβγ

FrontEnd`UnicodeCopy[Cell["asdasd\[Alpha]\[Beta]\[Gamma]"]]

asdasdαβγ

Or by Control-C:

FrontEnd`UnicodeCopy[Cell["asdasdαβγ"]]

(fixed, now, by doing a more selective replacement from UnicodeCharacters.tr)

One issue here is that I was a bit over zealous with some replacements when copying from cells and boxes (has to do with what comes out of the ExportPacket):

(Command-C): "asdasd\\[Alpha]\\[Beta]\\[Gamma]" 
(Control-C): "asdasd\α\β\γ" 

And just to include the original example:

\[Alpha]\[Beta] + 
 Mod[\[Delta]\[CapitalPsi], 2\[InvisibleTimes]\[Rho]^2]

becomes:

αβ + 
 Mod[δΨ, 2⁢ρ^2]

(note how, unfortunately, the added spacing from the cell copy is preserved)

b3m2a1
  • 46,870
  • 3
  • 92
  • 239
3

Edit: Updated code

As a generalization to the excellent answer by ens, Silvia's solution also can be added as a palette to the menu as follows. First, create and save the UniCodeCopy.m package, as described by ens. Then create as a separate notebook, perhaps named Unicode Copy Source.nb,

NotebookPut[
    Notebook[{Cell[BoxData[ButtonBox["\"UniCode Copy\"", Appearance -> Automatic,
        ButtonFunction :> Module[{codestr},                  
        AppendTo[$Path, FileNameJoin[{$UserDocumentsDirectory, 
            "Mathematica/MyPackages"}]]; 
        Needs["UniCodeCopy`"];                  
        FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]]; 
        codestr = (StringReplace[#1, {"\\\n" -> "", "\n" ~~ space : " " ... :> 
            StringJoin["\n", ConstantArray[space, 4]]}] & )
            [NotebookGet[ClipboardNotebook[]][[1, 1, 1]]]; 
            UniCodeCopy`WriteToClipboardUnicode[codestr]], 
            Evaluator -> Automatic, ImageSize -> 200, Method -> "Queued"]], 
            NotebookDefault]}, WindowSize -> {Fit, Fit}, 
            WindowMargins -> {{Automatic, 522}, {Automatic, 41}}, 
            WindowFrame -> "Palette", WindowElements -> {}, 
            StyleDefinitions -> "Palette.nb"], WindowTitle -> "UniCodeCopy"];
NotebookSave[%, FileNameJoin[{$UserDocumentsDirectory, 
    "Mathematica/MyPackages/UniCodeCopy.nb"}]]

Note that the third and fourth lines of code add the location of UniCodeCopy.m (in this case, my Mathematica/MyPackages directory) to $Path, if the location is not already there. (If it is there, these two lines of code can be omitted.) Executing Unicode Copy Source.nb creates the small palette, UnicodeCopy.nb, saved in Mathematica/MyPackages. Finally, use the Install Palette ... command in the Palette menu to register UnicodeCopy.nb as a palette. After Mathematica is restarted, it will display UnicodeCopy in the Palette menu.

bbgodfrey
  • 61,439
  • 17
  • 89
  • 156
2

Under windows with Nircmd available, one can add the code below (a modification of https://mathematica.meta.stackexchange.com/a/155/9754) to an initialization cell of
$UserBaseDirectory <> "\\Autoload\\FrontEnd\\init.m" to get Alt Gr+p as a shortcut for copying as unicode and having 4 spaces added at the beginning of each line.

FrontEndExecute[FrontEnd`AddMenuCommands["Copy", {MenuItem["Copy for MSE",
      FrontEndExecute[Module[{opts, content},
            If[MatchQ[NotebookRead[SelectedNotebook[]], {} | $Failed | NotebookRead[$Failed]], Beep[],

                opts = AbsoluteOptions[$FrontEndSession, "ExportTypesetOptions"];
                SetOptions[$FrontEndSession, "ExportTypesetOptions" -> {"PageWidth" -> 800, "CellBreaks" -> "\nCeLlBrEaK\n"}];
                FrontEndTokenExecute[SelectedNotebook[], "CopySpecial", "InputText"];
                content = NotebookGet[ClipboardNotebook[]];
                SetOptions[$FrontEndSession, opts];

                RunProcess[{"C:\\nircmd\\nircmdc.exe", "clipboard", "readfile", "\"" <> 
                          Export["C:\\nircmd\\mathCopy.txt", #, CharacterEncoding -> "Unicode"] <> "\""}] &[

                   Fold[StringReplace, StringJoin[{"    ", Cases[content, Cell[c_String, ___] :> c, All]}],
                      {{"" | "" -> "I", "" -> "E", "" | "" -> "<->", "\n" -> "\n    "},
                        {"    In[" ~~ NumberString ~~ "]:= " -> "    ",
                          "    CeLlBrEaK\n" | StartOfString ~~ "    Out[" ~~ NumberString ~~
                              "]= " ~~ e : Shortest[__] ~~ f : "\n    CeLlBrEaK" | EndOfString :>
                            "    (* " <> StringReplace[e, "\n    " -> "\n       "] <> " *)" <> StringTake[f, UpTo[5]],
                          "CeLlBrEaK" -> ""}}]]]]],
       MenuKey["p", Modifiers -> {"Control", "Command"}], System`MenuEvaluator -> Automatic, Method -> "Queued"]}]]
MeMyselfI
  • 1,116
  • 5
  • 12