What is in your Mathematica tool bag? [closed]

Question

We all know that Mathematica is great, but it also often lacks critical functionality. What kind of external packages / tools / resources do you use with Mathematica?

I'll edit (and invite anyone else to do so too) this main post to include resources which are focused on general applicability in scientific research and which as many people as possible will find useful. Feel free to contribute anything, even small code snippets (as I did below for a timing routine).

Also, undocumented and useful features in Mathematica 7 and beyond you found yourself, or dug up from some paper/site are most welcome.

Please include a short description or comment on why something is great or what utility it provides. If you link to books on Amazon with affiliate links please mention it, e.g., by putting your name after the link.

---

Packages:

1. LevelScheme is a package that greatly expands Mathematica's capability to produce good looking plots. I use it if not for anything else then for the much, much improved control over frame/axes ticks. Its newest version is called SciDraw, and it will be released sometime this year.
2. David Park's Presentation Package (US$50 - no charge for updates)
3. Jeremy Michelson's grassmannOps package provides resources for doing algebra and calculus with Grassmann variables and operators that have non trivial commutation relations.
4. John Brown's GrassmannAlgebra package and book for working with Grassmann and Clifford algebras.
5. RISC (Research Institute for Symbolic Computation) has a variety of packages for Mathematica (and other languages) available for download. In particular, there is Theorema for automated theorem proving, and the multitude of packages for symbolic summation, difference equations, etc. at the Algorithmic Combinatorics group's software page.

Tools:

1. MASH is Daniel Reeves's excellent Perl script essentially providing scripting support for Mathematica v7. (Now built in as of Mathematica 8 with the -script option.)
2. An alternate Mathematica shell with a GNU readline input (using python, *nix only)
3. ColourMaths package allows you to visually select parts of an expression and manipulate them. http://www.dbaileyconsultancy.co.uk/colour_maths/colour_maths.html

Resources:

1. Wolfram's own repository MathSource has a lot of useful if narrow notebooks for various applications. Also check out the other sections such as

   • Current Documentation,
   • Courseware for lectures,
   • and Demos for, well, demos.

2. The Mathematica Wikibook.

Books:

1. Mathematica programming: an advanced introduction by Leonid Shifrin (web, pdf) is a must read if you want to do anything more than For loops in Mathematica. We have the pleasure of having Leonid himself answering questions here.
2. Quantum Methods with Mathematica by James F. Feagin (amazon)
3. The Mathematica Book by Stephen Wolfram (amazon) (web)
4. Schaum's Outline (amazon)
5. Mathematica in Action by Stan Wagon (amazon) - 600 pages of neat examples and goes up to Mathematica version 7. Visualization techniques are especially good, you can see some of them on the author's Demonstrations Page.
6. Mathematica Programming Fundamentals by Richard Gaylord (pdf) - A good concise introduction to most of what you need to know about Mathematica programming.
7. Mathematica Cookbook by Sal Mangano published by O'Reilly 2010 832 pages. - Written in the well known O'Reilly Cookbook style: Problem - Solution. For intermediates.
8. Differential Equations with Mathematica, 3rd Ed. Elsevier 2004 Amsterdam by Martha L. Abell, James P. Braselton - 893 pages For beginners, learn solving DEs and Mathematica at the same time.

Undocumented (or scarcely documented) features:

7. How to customize Mathematica keyboard shortcuts. See this question.
8. How to inspect patterns and functions used by Mathematica's own functions. See this answer
9. How to achieve consistent size for GraphPlots in Mathematica? See this question.
10. How to produce documents and presentations with Mathematica. See this question.

Answer 1

Following function format[expr_] can be used to indent/format unformatted mathematica expressions that spans over a page

indent[str_String, ob_String, cb_String, delim_String] := 
  Module[{ind, indent, f, tab}, ind = 0; tab = "    ";
   indent[i_, tab_, nl_] := nl <> Nest[tab <> ToString[#] &, "", i];
   f[c_] := (indent[ind, "", " "] <> c <> indent[++ind, tab, "\n"]) /;StringMatchQ[ob, ___ ~~ c ~~ ___];
   f[c_] := (indent[--ind, "", " "] <> c <> indent[ind, tab, "\n"]) /;StringMatchQ[cb, ___ ~~ c ~~ ___];
   f[c_] := (c <> indent[ind, tab, "\n"]) /;StringMatchQ[delim, ___ ~~ c ~~ ___];
   f[c_] := c;
   f /@ Characters@str // StringJoin];
format[expr_] := indent[expr // InputForm // ToString, "[({", "])}", ";"];

(*    
format[Hold@Module[{ind, indent, f, tab}, ind = 0; tab = "    ";
 indent[i_, tab_, nl_] := nl <> Nest[tab <> ToString[#] &, "", i];
 f[c_] := (indent[ind, "", " "] <> c <> indent[++ind, tab, "\n"]) /;StringMatchQ[ob, ___ ~~ c ~~ ___];
 f[c_] := (indent[--ind, "", " "] <> c <> indent[ind, tab, "\n"]) /;StringMatchQ[cb, ___ ~~ c ~~ ___];
 f[c_] := (c <> indent[ind, tab, "\n"]) /;StringMatchQ[delim, ___ ~~ c ~~ ___];
 f[c_] := c;
 f /@ Characters@str // StringJoin]]
*)

---

ref: https://codegolf.stackexchange.com/questions/3088/indent-a-string-using-given-parentheses

Answer 2

This one was written by Alberto Di Lullo, (who doesn't appear to be on Stack Overflow).

CopyToClipboard, for Mathematica 7 (in Mathematica 8 it's built in)

CopyToClipboard[expr_] := 
  Module[{nb}, 
   nb = CreateDocument[Null, Visible -> False, WindowSelected -> True];
   NotebookWrite[nb, Cell[OutputFormData@expr], All];
   FrontEndExecute[FrontEndToken[nb, "Copy"]];
   NotebookClose@nb];

Original post: http://forums.wolfram.com/mathgroup/archive/2010/Jun/msg00148.html

I have found this routine useful for copying large real numbers to the clipboard in ordinary decimal form. E.g. CopyToClipboard["123456789.12345"]

Cell[OutputFormData@expr] neatly removes the quotes.

Answer 3

This is not a complete resource, so I'm throwing it here in the answers section, but I have found it very useful when figuring out speed issues (which, unfortunately, is a large part of what Mathematica programming is about).

timeAvg[func_] := Module[
{x = 0, y = 0, timeLimit = 0.1, p, q, iterTimes = Power[10, Range[0, 10]]},
Catch[
 If[(x = First[Timing[(y++; Do[func, {#}]);]]) > timeLimit,
    Throw[{x, y}]
    ] & /@ iterTimes
 ] /. {p_, q_} :> p/iterTimes[[q]]
];
Attributes[timeAvg] = {HoldAll};

Usage is then simply timeAvg@funcYouWantToTest.

EDIT: Mr. Wizard has provided a simpler version that does away with Throw and Catch and is a bit easier to parse:

SetAttributes[timeAvg, HoldFirst]
timeAvg[func_] := Do[If[# > 0.3, Return[#/5^i]] & @@ 
                     Timing @ Do[func, {5^i}]
                     ,{i, 0, 15}]

EDIT: Here's a version from acl (taken from here):

timeIt::usage = "timeIt[expr] gives the time taken to execute expr, \
  repeating as many times as necessary to achieve a total time of 1s";

SetAttributes[timeIt, HoldAll]
timeIt[expr_] := Module[{t = Timing[expr;][[1]], tries = 1},
  While[t < 1., tries *= 2; t = Timing[Do[expr, {tries}];][[1]];]; 
  t/tries]

Answer 4

Printing system symbol definitions without context prepended

The contextFreeDefinition[] function below will attempt to print the definition of a symbol without the most common context prepended. The definition then can be copied to Workbench and formatted for readability (select it, right click, Source -> Format)

Clear[commonestContexts, contextFreeDefinition]

commonestContexts[sym_Symbol, n_: 1] := Quiet[
  Commonest[
   Cases[Level[DownValues[sym], {-1}, HoldComplete], 
    s_Symbol /; FreeQ[$ContextPath, Context[s]] :> Context[s]], n],
  Commonest::dstlms]

contextFreeDefinition::contexts = "Not showing the following contexts: `1`";

contextFreeDefinition[sym_Symbol, contexts_List] := 
 (If[contexts =!= {}, Message[contextFreeDefinition::contexts, contexts]];
  Internal`InheritedBlock[{sym}, ClearAttributes[sym, ReadProtected];
   Block[{$ContextPath = Join[$ContextPath, contexts]}, 
    Print@InputForm[FullDefinition[sym]]]])

contextFreeDefinition[sym_Symbol, context_String] := 
 contextFreeDefinition[sym, {context}]

contextFreeDefinition[sym_Symbol] := 
 contextFreeDefinition[sym, commonestContexts[sym]]

---

withRules[]

Caveat: This function does not localize variables the same way With and Module do, which means that nested localization constructs won't work as expected. withRules[{a -> 1, b -> 2}, With[{a=3}, b_ :> b]] will replace a and b in the nested With and Rule, while With doesn't do this.

This is a variant of With that uses rules instead of = and :=:

ClearAll[withRules]
SetAttributes[withRules, HoldAll]
withRules[rules_, expr_] :=
  Internal`InheritedBlock[
    {Rule, RuleDelayed},
    SetAttributes[{Rule, RuleDelayed}, HoldFirst];
    Unevaluated[expr] /. rules
  ]

I found this useful while cleaning up code written during experimentation and localizing variables. Occasionally I end up with parameter lists in the form of {par1 -> 1.1, par2 -> 2.2}. With withRules parameter values are easy to inject into code previously written using global variables.

Usage is just like With:

withRules[
  {a -> 1, b -> 2},
  a+b
]

---

Antialiasing 3D graphics

This is a very simple technique to antialias 3D graphics even if your graphics hardware doesn't support it natively.

antialias[g_, n_: 3] := 
  ImageResize[Rasterize[g, "Image", ImageResolution -> n 72], Scaled[1/n]]

Here's an example:

Note that a large value for n or a large image size tends to expose graphics driver bugs or introduce artefacts.

---

Notebook diff functionality

Notebook diff functionality is available in the <<AuthorTools` package, and (at least in version 8) in the undocumented NotebookTools` context. This is a little GUI to diff two notebooks that are currently open:

PaletteNotebook@DynamicModule[
  {nb1, nb2}, 
  Dynamic@Column[
    {PopupMenu[Dynamic[nb1], 
      Thread[Notebooks[] -> NotebookTools`NotebookName /@ Notebooks[]]], 
     PopupMenu[Dynamic[nb2], 
      Thread[Notebooks[] -> NotebookTools`NotebookName /@ Notebooks[]]], 
     Button["Show differences", 
      CreateDocument@NotebookTools`NotebookDiff[nb1, nb2]]}]
  ]

Answer 5

This code makes a palette that uploads the selection to Stack Exchange as an image. On Windows, an extra button is provided that gives a more faithful rendering of the selection.

Copy the code into a notebook cell and evaluate. Then pop out the palette from the output, and install it using Palettes -> Install Palette...

If you have any trouble with it, post a comment here. Download the notebook version here.

---

Begin["SOUploader`"];

Global`palette = PaletteNotebook@DynamicModule[{},

   Column[{
     Button["Upload to SE",
      With[{img = rasterizeSelection1[]},
       If[img === $Failed, Beep[], uploadWithPreview[img]]],
      Appearance -> "Palette"],

     If[$OperatingSystem === "Windows",

      Button["Upload to SE (pp)",
       With[{img = rasterizeSelection2[]},
        If[img === $Failed, Beep[], uploadWithPreview[img]]],
       Appearance -> "Palette"],

      Unevaluated@Sequence[]
      ]
     }],

   (* Init start *)
   Initialization :>
    (

     stackImage::httperr = "Server returned respose code: `1`";
     stackImage::err = "Server returner error: `1`";

     stackImage[g_] :=
      Module[
       {getVal, url, client, method, data, partSource, part, entity,
        code, response, error, result},

       getVal[res_, key_String] :=
        With[{k = "var " <> key <> " = "},
         StringTrim[

          First@StringCases[
            First@Select[res, StringMatchQ[#, k ~~ ___] &],
            k ~~ v___ ~~ ";" :> v],
          "'"]
         ];

       data = ExportString[g, "PNG"];

       JLink`JavaBlock[
        url = "http://stackoverflow.com/upload/image";
        client =
         JLink`JavaNew["org.apache.commons.httpclient.HttpClient"];
        method =
         JLink`JavaNew[
          "org.apache.commons.httpclient.methods.PostMethod", url];
        partSource =
         JLink`JavaNew[
          "org.apache.commons.httpclient.methods.multipart.\
ByteArrayPartSource", "mmagraphics.png",
          JLink`MakeJavaObject[data]@toCharArray[]];
        part =
         JLink`JavaNew[
          "org.apache.commons.httpclient.methods.multipart.FilePart",
          "name", partSource];
        part@setContentType["image/png"];
        entity =
         JLink`JavaNew[
          "org.apache.commons.httpclient.methods.multipart.\
MultipartRequestEntity", {part}, method@getParams[]];
        method@setRequestEntity[entity];
        code = client@executeMethod[method];
        response = method@getResponseBodyAsString[];
        ];

       If[code =!= 200, Message[stackImage::httperr, code];
        Return[$Failed]];
       response = StringTrim /@ StringSplit[response, "\n"];

       error = getVal[response, "error"];
       result = getVal[response, "result"];
       If[StringMatchQ[result, "http*"],
        result,
        Message[stackImage::err, error]; $Failed]
       ];

     stackMarkdown[g_] :=
      "![Mathematica graphics](" <> stackImage[g] <> ")";

     stackCopyMarkdown[g_] := Module[{nb, markdown},
       markdown = Check[stackMarkdown[g], $Failed];
       If[markdown =!= $Failed,
        nb = NotebookCreate[Visible -> False];
        NotebookWrite[nb, Cell[markdown, "Text"]];
        SelectionMove[nb, All, Notebook];
        FrontEndTokenExecute[nb, "Copy"];
        NotebookClose[nb];
        ]
       ];

     (* Returns available vertical screen space,
     taking into account screen elements like the taskbar and menu *)


     screenHeight[] := -Subtract @@
        Part[ScreenRectangle /. Options[$FrontEnd, ScreenRectangle],
         2];

     uploadWithPreview[img_Image] :=
      CreateDialog[
       Column[{
         Style["Upload image to the Stack Exchange network?", Bold],
         Pane[

          Image[img, Magnification -> 1], {Automatic,
           Min[screenHeight[] - 140, 1 + ImageDimensions[img][[2]]]},
          Scrollbars -> Automatic, AppearanceElements -> {},
          ImageMargins -> 0
          ],
         Item[
          ChoiceButtons[{"Upload and copy MarkDown"}, \
{stackCopyMarkdown[img]; DialogReturn[]}], Alignment -> Right]
         }],
       WindowTitle -> "Upload image to Stack Exchange?"
       ];

     (* Multiplatform, fixed-width version.
        The default max width is 650 to fit Stack Exchange *)
     rasterizeSelection1[maxWidth_: 650] :=
      Module[{target, selection, image},
       selection = NotebookRead[SelectedNotebook[]];
       If[MemberQ[Hold[{}, $Failed, NotebookRead[$Failed]], selection],

        $Failed, (* There was nothing selected *)

        target =
         CreateDocument[{}, WindowSelected -> False, Visible -> False,
           WindowSize -> maxWidth];
        NotebookWrite[target, selection];
        image = Rasterize[target, "Image"];
        NotebookClose[target];
        image
        ]
       ];

     (* Windows-only pixel perfect version *)
     rasterizeSelection2[] :=
      If[
       MemberQ[Hold[{}, $Failed, NotebookRead[$Failed]],
        NotebookRead[SelectedNotebook[]]],

       $Failed, (* There was nothing selected *)

       Module[{tag},
        FrontEndExecute[
         FrontEndToken[FrontEnd`SelectedNotebook[], "CopySpecial",
          "MGF"]];
        Catch[
         NotebookGet@ClipboardNotebook[] /.
          r_RasterBox :>
           Block[{},
            Throw[Image[First[r], "Byte", ColorSpace -> "RGB"], tag] /;
              True];
         $Failed,
         tag
         ]
        ]
       ];
     )
   (* Init end *)
   ]

End[];

Answer 6

I'm sure a lot of people have encountered the situation where they run some stuff, realizing it not only stuck the program, but they also haven't saved for the last 10 minutes!

EDIT

After suffering from this for some time, I one day found out that one can create auto-save from within the Mathematica code. I think that using such auto-save have helped me a lot in the past, and I always felt that the possibility itself was something that not a lot of people are aware that they can do.

The original code I used is at the bottom. Thanks to the comments I've found out that it is problematic, and that it is much better to do it in an alternative way, using ScheduledTask (which will work only in Mathematica 8).

Code for this can be found in this answer from Sjoerd C. de Vries (Since I'm not sure if it's OK to copy it to here, I'm leaving it as a link only.)

---

The solution below is using Dynamic. It will save the notebook every 60 seconds, but apparently only if its cell is visible. I'm leaving it here only for completion reasons. (and for users of Mathematica 6 and 7)

/EDIT

To solve it I use this code in the beginning of a notebook:

Dynamic[Refresh[NotebookSave[]; DateString[], UpdateInterval -> 60]]

This will save your work every 60 seconds.
I prefer it to NotebookAutoSave[] because it saves before the input is processed, and because some files are more text than input.

I originally found it here: http://en.wikipedia.org/wiki/Talk:Mathematica#Criticisms

Note that once running this line, saving will happen even if you close and re-open your file (as long as dynamic updating is enabled).

Also, since there is no undo in Mathematica, be careful not to delete all your content, since saving will make it irreversible (as a precaution move, I remove this code from every finished notebook)