Bug in WebExecute TypeElement with diacritics characters [Solved]

Question

[Solved in 12.1.1]

When using WebExecute TypeElement, I get $Failed if I have a diacritics character in the submitted string. Here is a minimal example:

session = StartWebSession["Chrome"];
WebExecute["OpenPage"->"https://stackoverflow.com/questions/35126293/how-to-set-text-into-textarea-with-selenium-webdriver"];
textArea = First@WebExecute["LocateElements"->"XPath"->"//*[@id=\"wmd-input\"]"];
WebExecute["TypeElement"->{textArea, "à"}]

If I change "à" to "a", it works so far so good.

Any clue?

crosspost in Wolfram Community

Answer 1

Kuba is right with his solution and we need to patch this in the WebUnit code.

Everything in the WebUnit code eventually runs through a single fetch function which makes the browser command request to the web-driver executable (a small binary which controls the browser).

In my personal experience these requests work most reliable by switching to byte arrays, so this is an initial draft of a solution I am going to propose to fix the issue.

ClearAll[WebUnit`Private`fetch];

WebUnit`Private`fetch[
 WebUnit`Private`sessioninfo_,
 WebUnit`Private`type_, 
 WebUnit`Private`path_, 
 WebUnit`Private`data_,
 WebUnit`Private`key_
] := Module[{
 WebUnit`Private`res, 
 WebUnit`Private`fetchResult
},
 WebUnit`Private`fetchResult = URLRead[
  HTTPRequest[
   WebUnit`Private`sessioninfo[[3]] <> WebUnit`Private`path, <|
    "Method" -> WebUnit`Private`type,
    "Body" -> WebUnit`Private`json[WebUnit`Private`data]
   |>
  ], "BodyByteArray"
 ]; 
 If[
  !ByteArrayQ[WebUnit`Private`fetchResult], 
  Return[$Failed]
 ]; 
 If[
  ImportByteArray[WebUnit`Private`fetchResult, "Text"] === "missing command parameters",
  Return[$Failed]
 ];
 WebUnit`Private`res = ImportByteArray[WebUnit`Private`fetchResult, "JSON"]; 
 WebUnit`Private`DebugPrint[ WebUnit`Private`format[WebUnit`Private`res] ]; 
 WebUnit`Private`key /. WebUnit`Private`res
]

As far as I can tell this also fixes your problem and I also tested it with this example:

b = StartWebSession[]
WebExecute[b, "OpenWebPage" -> "https://www.google.com"]
WebExecute[b, "JavascriptExecute" -> "document.getElementsByName('q')[0].value='bábáç'"]

Answer 2

Full answer and explanation probably involves a lot of fun time tracking encoding performed internally. And I won't spend it that way again.

You can use PrintDeifinitions or

Trace[ WebExecute["TypeElement" -> {textArea, text_}], _URLFetch]

To see that the call is:

URLFetch["http://localhost:{port}/session/{sessionId}/element/{elementId}/value",
  "Method" -> "Post", "BodyData" -> "{
    \"value\":[
        characters...
    ]
  }"]

Something goes wrong and characters are not encoded correctly nor the response is parsed correctly.

Here is a quick workaround:

typeElementPath // ClearAll

typeElementPath[elem : _WebElementObject, text_] := Module[{url},

  url = URLBuild[
    <| URLParse @ elem["URL"], 
     "Path" -> {"session", elem["SessionID"], "element", elem["ElementId"]}|>
  ];

  URLExecute[
   HTTPRequest[
    url <> "/clear",
    <| "Method" -> "Post"|> 
   ]   
  ];

  URLExecute[
   HTTPRequest[
    url <> "/value",
    <|
     "Method" -> "Post",
     "Body" -> ExportString[<|"value" -> Characters@text|>, "RawJSON", "Compact" -> True]
     |>,
    CharacterEncoding -> None
    ],
   "RawJSON"
   ]
  ]

typeElementPath[textArea, "αó"]

<|"sessionId" -> "a0c4c6e971d1506dd115351a76278480", "status" -> 0,  "value" -> Null|>

You can skip the first URLExecute to append instead of overwrite the element.

This should work for IE/Chrome/Edge. For Firefox you probably just need to use:

 ExportString[<|"text" -> text|>, "RawJSON", "Compact" -> True]

But I have not tested it.