5

I'm trying to build my first paclet which includes a .txt data file. As per documentation, I added {"Path"} to "Extensions" section in PacletInfo.wl, so that it now looks like this:

PacletObject[
    <|
        "Name" -> "WordGrid",
        "Version" -> "0.0.1",
        "WolframVersion" -> "13+",
        "Extensions" ->
            {
                {
                    "Path"
                },
                {
                    "Kernel",
                    "Root" -> "Kernel",
                    "Context" -> "WordGrid`"
                },
                {
                    "Documentation",
                    "Language" -> "English"
                }
            }
    |>
]

I put the data file under the paclet's root directory in WordGrid/rusian_nouns.txt and am now trying to access it from within the paclet with e.g. Import["WordGrid/russian_nouns.txt"]:

RussianNouns[] := RussianNouns[] =
    ToLowerCase /@ StringSplit[Import["WordGrid/russian_nouns.txt"]];

Everything seems to work during the development: the following code, executed in a fresh kernel, returns the list of Russian nouns as expected:

PacletDirectoryLoad[NotebookDirectory[] <> "WordGrid"]
Needs["WordGrid`"]
RussianNouns[]

However, after I build the paclet with

Needs["PacletTools`"]
PacletBuild[NotebookDirectory[] <> "WordGrid"]

I can't see the .txt file in the build directory or the manifest, and the paclet file itself is too small to contain my list of words.

What am I doing wrong?

Victor K.
  • 5,146
  • 3
  • 21
  • 34
  • Looks like a bug in BuildPacket: when I create the paclet using CreatePacletArchive and install the result, everything works as expected. – Victor K. Oct 16 '22 at 20:04
  • I think you're supposed to use "Resource" extension, then it will be included in the build file. Specifying empty root Root->"" with "Path" extension will enable you to use it with Import. Also you usually name each resource (i.e. "RussianNouns") and path to it can be extracted with PacletObject["WordGrid"]["AssetLocation", "RussianNouns"] (in earlier versions it was PacletResource, but it got scrubbed it seems). – swish Oct 16 '22 at 20:39
  • Thanks @swish for the answer! "Resource" extension is undocumented; did you mean "Asset"? In the documentation link I posted above, that's what is suggested. For my needs, I want to avoid explicit resource/asset enumeration.

    Also, the documentation, in the subsection "Path", mentions nothing about the "Root" key - everything should work as I describe above.

     – Victor K. Oct 16 '22 at 20:55
  • Maybe I need to include "Root"->"" indeed, but in that case the documentation (and some videos like https://www.wolfram.com/broadcast/video.php?v=3537) should probably be updated - e.g., in the link above, at 13:20 they show usage of "Path" without "Root" and at 16:42 the expected behavior of ReadList. – Victor K. Oct 16 '22 at 20:58
  • "Resource" extension is described a little bit here. Here is an example of using it in the wild. – swish Oct 16 '22 at 21:26
  • 1
    I think "Asset" and "Resource" are the same thing and can be used both, not sure why official documentation uses one, but all the developers use another one though :) – swish Oct 16 '22 at 21:31
  • I'm not an insider, but could it be legacy code? At least in the official documentation, it's recommended to use PacletObject with an Association as an argument and not plain Paclet anymore... – Victor K. Oct 16 '22 at 21:41
  • The PacletTools itself also is a good example: SystemOpen@FileNameJoin[{PacletObject["PacletTools"]["Location"],"PacletInfo.wl"}]. And it uses old Paclet format, but with new "Asset" extension. I would recommend following the documentation as much as possible. – swish Oct 16 '22 at 21:51
  • Yep, it looks like "Resource" is much more common than "Asset", and "Path" is not used often (but is used some):

    With[{all = PacletFind[<|"WolframVersion" -> All|>]}, Length@Select[all, x |-> MemberQ[x["Extensions"], #, All]] & /@ {"Path", "Asset", "Resource", _}] returns {3, 4, 84, 350}.

     – Victor K. Oct 17 '22 at 02:50

1 Answers1

2

I'm the developer of PacletBuild. This behavior is intentional, though I can understand why it seems like a bug.

You can fix this by moving any files you intend to load via file path name lookup into a subdirectory (name doesn't matter), and modifying the "Path" extension to point at that subdirectory. For example:

  • File: WordGrid/Files/russian_nouns.txt
  • Extension: {"Path", "Root" -> "Files"}
  • Lookup code: Import["WordGrid/russian_nouns.txt"] (unchanged)

The issue is that PacletBuild uses PacletExtensionFiles to determine what files are associated with each entry in the paclet "Extensions" list, and that "Path" extensions that use the implicit default "Root" -> "." have no well-defined definition for that operation.

For example, consider "Asset" extensions. The associated files of an "Asset" extension are obvious and explicit: they are simply whatever files are specified in the "Asset" extension options.

However, the "Path" extension is a bit of an unusual case, because there is no way to statically determine what files the user intends to refer to via path lookup, which means that potentially any file in the entire paclet directory could be named.

PacletExtensionFiles makes the assumption that every file is associated with a particular extension; without specifying a custom "Root" location, "Path" extensions violate that assumption.

By specifying a "Root" location that isn't simply the entire paclet's root directory, PacletExtensionFiles accepts that more narrow subdirectory as being unambiguous.

Edit: For an illustration of the above, here is the result of PacletExtensionFiles[PacletObject["ExamplePaclet"], "Path"] on a paclet with a {"Path"} extension:

<|{"Path", <||>} -> Missing["NotAvailable"]|>

vs. with a {"Path", "Root" -> "Files"} extension:

<|{"Path", <|"Root" -> "Files"|>} -> {"/Users/connorgray/Desktop/ExamplePaclet/Files/Data.wl"}|>

PacletBuild should arguably issue a warning about this issue, and suggest the fix I describe above.

Connor Gray
  • 421
  • 4
  • 7
  • Thank you Connor for a detailed answer! May I suggest that the documentation should be modified accordingly? My intuition about how Path extension should behave is based solely on the examples I found in the standard documentation and in various YouTube videos, all of which show {"Path"} without an explicit Root option. – Victor K. Oct 30 '22 at 03:22
  • 1
    @VictorK. That is a reasonable suggestion. I can also update PacletBuild to issue a clear warning about this behavior, so users won't silently have missing files, like you ran into. – Connor Gray Oct 31 '22 at 21:50
  • Thanks! Btw, one other thought I have is that the additional source of confusion is the behavior of CreatePacletArchive, which does include all files when Path is enabled even without the Root option (but looks like you are going to eventually deprecate it?). – Victor K. Oct 31 '22 at 23:14
  • 1
    @VictorK. I just wanted to give you an update that this is fixed in the upcoming WL v13.2.1 bugfix release. In that version, PacletBuild[..] will return an error when building a paclet with a {"Path"} extension. The fix is as described in my answer: adding a "Root" -> "Something" value and putting the desired files in that directory. – Connor Gray Jan 04 '23 at 23:23
  • 1
    @VictorK. (Oops, didn't mean to post that previous comment so soon.)

    Regarding CreatePacletArchive, you can think of this as a lower-level function that simply bundles up a collection of files into the archive format used by .paclet files. CreatePacletArchive is analogous to Compress is that it is mostly agnostic as to the content of the directory being archived. PacletBuild[..] is the higher-level function intended for most use-cases.

     – Connor Gray Jan 04 '23 at 23:25
  • Perfect, thanks @ConnorGray for providing an update, and of course for fixing it in the first place! – Victor K. Jan 05 '23 at 23:30