The following software list is idiosyncratic. I have tried to make it generally useful for economists. I favor cross-platform solutions, since I view the current near monopoly of the end-user operating system as a momentary historical aberration. Suggested changes and additions are welcome. A resource complementary to this page is RFE's Software page.
Serious coding calls for a serious text editor! All other things equal, multi-platform solutions are always preferable.
I recommend Vim. It is highly configurable, available on essentially every platform and quite wonderfully designed. On top of all this, it is charityware: you pay nothing for this excellent software, but users are encouraged to make a donation for needy children in Uganda. However, many good choice exist, including the very closely related NeoVim.
Mode-based editing rocks! However, it also takes practice. If you choose Vim or NeoVim (recommended!), be sure to do the tutorial (or possibly this online tutorial).
See Vim user hints.
Python users may want to consider the interesting and powerful Python Tools for Visual Studio in Visual Studio Code. (There is additionally a Vim editing mode.)
The elephant in the room is still GNU emacs: big, powerful, and free. (Vincent Goulet offers a modified emacs that is easy to install across platforms and is particularly useful for LaTeX users.) Some prefer the closely related XEmacs (but development on that has slowed). Many users will find the somewhat less powerful but much smaller mg3a provides sufficient emacs style functionality.
Aside: you can still get the traditional vi editor, which has been free and open source since 2002.
Once you have chosen your editor, find a decent monospaced font. (Crucial for programming; always examine this character sequence: Il1aoe0O.) My favorite is Andale Mono, a TrueType font that I find exceptionally readable even at small font sizes. There appears to be more than one legal yet free distribution, due to the original licensing terms, but you can also purchase the font. My second choice is the Unicode DejaVu font family. (Derivative from Bitstream's Vera Fonts; available as a Latin-Cyrillic-Greek subset; License) Many people like the free and open source Inconsolata.
\pdfmapfile{=stix.map} or globally by following the
instructions in the documentation
\documentclass{article}
%\pdfmapfile{=stix.map} %if the map file is not activated globally
\usepackage{stix}
\begin{document}
Some text, and a math formula \(a+b=\sqrt{c}\).
\end{document}
A few more freely available mono-space fonts ... Google’s Noto Sans Mono is getting very favorable reviews. Most systems include some version of CourierNew, which is mostly adequate, or LucidaConsole, which is pretty good. (Unfortunately, neither adequately distinguishes 0 from O.) Another good choice from Adobe is Source Code Pro,which in regular weight looks quite a bit like Andale Mono, but which somewhat overweights its boldface font. (OpenType, Latin-1 only; License: Open Font License 1.1) Another popular font is Envy Code R. (True Type, Latin-1 and Latin-1 Supplement; License: gratis but not redistributable) For more choices, filter on mono-spaced fonts at Google Fonts or Font Squirrel. User ratings of “programming fonts” can be found on Slant. The GNU FreeFont mono-space fonts are available as OpenType or TrueType. Some people like Anonymous (OFL). There is also a more narrowly focused but very useful APL font (SImPL.TTF). Those needing a small font-size might consider the ProFont bitmap font, which looks great at 9pt. (License: MIT)
Gentium Plus is a high-quality FOSS unicode font set with some math-symbol support.
The Arev fonts are designed to be useful for presentations containing math.
The most natural approach to spell checking is modular, but many editors nevertheless include their own spell checker. If yours does not, there are many free options. Perhaps the most obvious cross-platform option is Aspell. (If you will use the Windows release, see Aspell hints.) Aspell derives from the more famous ispell. (There seem to be separate implementations of ispell for Unix and OS/2 and ispell for Win95/NT. You can also try jspell in a DOS environment.) If you need a spell checker for general Windows applications, try All-Purpose Spell Checker or Spell Checker for Edit Boxes. (They are free.)
Vim 7+ includes a spell checker. You can also integrate Vim with aspell, which is my preference.
These days, your favorite dictionaries and thesauri are probably available free online.
My favorite crowd-sourced dictionary is Wiktionary,
which is remarkably good, even for etymology.
My favorite commercial dictionary is The American Heritage® Dictionary of the English Language.
The DICT Development Group offers a free online dictionary that provides a fast, convenient
interface to the Collaborative International Dictionary of English.
In addition,
you can submit a query for "yourword" directly as
http://www.dict.org/bin/Dict?Form=Dict1&Query=yourword&Strategy=*&Database=*.
Windows users can download for free the Freelang translation dictionaries.
Aiksaurus is a powerful, free, cross-platform thesaurus. Windows users can download WordWeb for free.
The only stream editor I have experience with is sed, which is very fast, powerful, and cross platform. (License: GPL) For an introductory tutorial, try SED: A Non-interactive Text Editor. If you decide to use sed a lot, you may wish to try sedsed, and Python-based sed debugger.
Scientific documents will generally include typeset mathematics. Proper typesetting aids communication, and ISO31/XI therefore provides typesetting standards. (See Claudio Beccari's discussion, but also the thread on StackExchange.)
A good cross platform solution is Open Office, which includes the oomath formula editor. Open Office is free and open source software.
Another cross platform solution is AbiWord, which brilliantly uses MathML as its internal representation of mathematics. AbiMath can import MathML equations. AbiWord also support itex for equation input, which is quite wonderful.
For Windows users, the most common approach is Microsoft Word with embedded OfficeMath equations. Unfortunately for old documents, new versions of Word no longer support Equation Editor 3 or earlier. Unfortunately, many journals do not allow embedded equations in submitted articles. This is a rather clumsy environment if your writing includes much mathematics, but it is widely used.
Some working environments allow you to see your mathematics in a form closely related to how it will be printed, while reserving polished document formatting to a separate "typesetting" process. Anyone who has printed a web page is familiar with the difference between the screen display and the typeset printed version (or "print preview") of the document: just imagine directly editing such a screen display before typesetting your document. These are sometimes called WYSIWYM (what you see is what you mean) environments.
The most popular WYSIWYM environments are interfaces to an underlying LaTeX document.
In such an environment,
you will see for example the glyph β instead of the LaTeX code \beta.
The only cross-platform implementation I know of is LyX. Binaries are available for Linux, Mac, and Windows users. LyX can interface with the Maxima computer algebra system. (Just add the path to Maxima in Tools -> Preferences -> Paths.)
An interesting alternative is TeXmacs.
Windows users appear to have a number of commercial choices, the most popular of which seems to be Scientific Notebook and Scientific Word. (Scientific Notebook is not marketed as a LaTeX front end, but as of 2004 it still saves in LaTeX format and with a tiny amount of editing can be used that way.) See Scientific Word user hints.
Mathematica notebooks can also be used as an WYSIWYM interface for mathematical documents, with some support for both LaTeX and MathML export.
In the WYSIWYM world, MathML is an obvious alternative to the LaTeX as an underlying document format. (In principle, this should not matter much to a user, who should be able to work in the GUI environment and not worry much about the underlying format.) The Amaya browser/authoring tool enables the WYSIWYM production of mathematical documents, uses MathML as the underlying format. Unfortunately, I find Amaya slow and clumsy for the production of mathematics, and development has stopped.
The most powerful way to produce mathematics is to learn a markup language for type-setting mathematics. In this case, what you see is your markup (WYSIYM). This also provides a big advantage for the process of writing: you get to use your favorite text editor and thereby get access to powerful editing facilities. The most popular choice is LaTeX, which is almost the universal WYSIYM choice among mathematicians, physicists, and economists. It is also my personal choice. An alternative is Lout, which is much less widely used but has some strong advocates. A full Lout installation is remarkably small: the download is less than 2Mb. The math in a Lout source document looks a lot like math in the EQN language. (Speaking of which, some people insist that EQN and GROFF remain the right way to typeset mathematics.)
If you compile your LaTeX documents with LuaLaTeX, you can use the unicode-math package. This allows you to enter math as UTF-8 characters, in addition to using the usual LaTeX codes. This produces much more readable documents, but only a few fonts have substantial support for the Unicode math characters. (The unicode-math documentation includes a list of fonts, including TeX Gyre Pagelle Math and Asana Math.)
There are a number of efforts to provide a meta-format from which both LaTeX and XHTML/MathML documents can be generated. The most notable and currently developed is probably GELLMU, although tbook is also interesting. I like the much simpler restructured text (reST), which allows inline and display math in LaTeX format. (See my reST hints.) Citation handling is currently (2005) weak, but bibstuff provides some basic capabilities. Math support is improving. It may also be worth trying reST plus Mozilla's itex for web math. Other metaformat options include: Almost Free Text (AFT), MarkDown, TextTile, and setext.
reST to HTML: just use the rst2html writer. If you have included math in the document, use the rst2mathml writer.
The usual way of creating PDF from an reST document is the following: use the rst2latex writer (or the rst2latexmath writer) to convert the document to LaTeX format, and then use pdftex to produce the PDF.
rst2pdf produces PDF directly using the ReportLab library. Good math handling via Matplotlib's mathtext!
You can also produce HTML with rst2html and then convert it. Using Adobe Acrobat can work moderately well, either by printing from a browser or by directly importing the HTML document. Alternatively, you can use the xhtml2pdf converter (License: GPL or commercial) or the Prince XML converter.
Background: LaTeX is a document formatting package based on TeX. TeX is typesetting software, arguably the best in the world (especially for mathematics). In this section I will concentrate on LaTeX and related resources.
To get started with LaTeX you will need two things: a text editor or IDE, and a LaTeX distribution (such as MikTeX, OzTeX, teTeX, or fptex). Amazingly enough, you can get everything you need for free. You can also get great help from other users, e.g. on comp.text.tex or the MikTeX users list.
AU Students:
For mathematical documents and for long documents (like dissertations!) you should use the LaTeX document preparation system.
There is an AU Thesis Class,
which automatically handles all the formatting of your dissertation or thesis.
Instructions for using this with Scientific Word are included.
Also see MacKichan Software's basic instructions and
detailed instructions.
You can also use this custom document class
with LyX.
That said,
my recommendation is to us Vim with my MiKTeX menus,
or with Vim-LaTeX.
If you are used to word processors,
you may initially find the process of creating a LaTeX document a bit clumsy:
you do not see the document formatting as you write.
However for long documents this is actually an advantage,
which you will grow to appreciate.
You will create your LaTeX input file with your text editor.
In addition to your text,
you will add certain LaTeX formatting commands to this input file.
You will then use your LaTeX distribution to process this input file,
which will produce a device independent (.dvi) file that you can view and print.
Details follow.
One caveat:
while you are working on an academic article,
LaTeX is an excellent environment.
But when submission time comes,
you may pick a journal that does not accept LaTeX.
What to do?
i. Article format generally does not matter until acceptance:
hardcopy submissions are still usually expected,
and otherwise PDF is generally acceptable.
So there is really no problem until an article is accepted at such a journal.
ii. Most journals that express a preference either ask for standard LaTeX or ask for common word-processor formats, but even in the latter case LaTeX is usually acceptable.
iii. If you have written an article in LaTeX and it has been accepted by a journal that for some reason will not accept LaTeX,
you can easily convert your article to a different format.
MiKTeX tools have been released for UN*X systems.
MikTeX is currently the most widely used LaTeX distribution on the AU campus. We are usually behind a version or two.
TeX Live provides a comprehensive TeX system for most systems. BaKoMa TeX is an inexpensive distribution for Win95 that got some good press, and it includes PostScript versions of the Computer modern fonts. (A very nice feature.) You can purchase a TeX system (with a lot of other useful stuff) from NTG as 4allTeX.Novices can get acquainted with LaTeX by looking at a tutorial. Once you are using some version of LaTeX, you are likely to find LaTeX: A Document Preparation System by L. Lamport to be indispensable. There are many other good books on TeX and LaTeX. The free lshort2e.pdf has most of what a novice will need. (Check for the most recent version.) More detail can be found in Making TeX Work and many other excellent books. If you need to write your own class (and you probably do not!) try clsguide.pdf.
Users with questions should check the LaTeX FAQ, and the LaTeX homepage. If you are sure you questions are not answered there, ask on the comp.text.tex newsgroup. CTAN and the CTAN-Web, also offer a wealth of information about TeX and LaTeX. (Be sure to visit Graham Williams's catalogue of information.) You may also find some useful TeX Information at UCC. If you get truly ambitious, Victor Eijkhout's TeX by Topic is now available online. (Most users will never need such a detailed understanding of TeX, however.)
LaTeX provides a basic picture environment that is pretty powerful. Still, you will often need more. You can easily include PDF, JPG, or PNG images in your documents. See Graphics for LaTeX for more details.
See the more complete list at http://www.loria.fr/services/tex/english/outils.html.
Extensive conversion capabilities may be provided by pandoc. (Language: Haskell; License: GPL) See this blog for a LaTeX-to-Word example. One interesting possibility is plasTeX, which parses LaTeX for output in various formats.
If you use LyX, there is a LyX (via LaTeX) to OpenOffice format converter. (See the same URL for a Mac front end.)
PDF files can be read by anyone with access to a PDF viewer. I have had excellent results using pdflatex. But see Markus Kuhn's tips on how to use (La)TeX to effectively produce highest-quality PDF files.
Note that a successful translation into HTML or MathML can subsequently be read by any modern word processor. (Of course it will not look as good as in LaTeX!)
for %i in (*.eps) do epstopdf --gsopt=-dPDFSETTINGS=/prepress %i.
Windows users may be interested in the small, fast RoPS interpreter,
which has a free Language Level 1 version.
(RoPS source code appears to be available.)
Serious PostScript programmers may profit from PSAlter's helpful debugging facilities.
A fuller discussion of options can be found in Wiedman on Screen Presentation Tools. Here are some options using reStructuredText.
- rst2pdf http://lateral.netmanagers.com.ar/stories/BBS52.html (Good math handling via Matplotlib's mathtext!) - rst2s5 http://docutils.sourceforge.net/docs/user/slide-shows.html http://docutils.sourceforge.net/docs/user/slide-shows.html http://www.python.org/doc/slideshows/5min-reST-demo.txt - Bruce http://pypi.python.org/pypi/bruce http://sites.google.com/site/r1chardj0n3s/howto (You can embed a Python interpreter in your presentation page!) - rst2odp http://panela.blog-city.com/rst2odp_on_pypi.htm - rst2beamer http://www.agapow.net/programming/python/rst2beamer http://home.cs.siue.edu/~rkrauss/python_website/ - rst2slidy (cannot find it, but here's Slidy: http://www.w3.org/Talks/Tools/Slidy )
Roberto Alesina mentions that if you choose rst2pdf then fancy transitions are hard, handouts have to be a separate document, there is no way to easily center something vertically, and you need to create different versions for different screen aspect ratios. (On the other hand, everything end up in a single file, your PDF can play almost anywhere, and you know exactly how it will look.)
If you know LaTeX, the beamer is great, and you can compile beamer presentation directly with pdftex. (I found getting started with the beamer document class using MiKTeX's pdflatex to be completely painless.) Some people prefer powerdot (which supercedes Prosper). With powerdot you will need to go .tex to .dvi to .ps to .pdf. (Many integrated TeX environments can automatically implement that sequence for you.) Some people like the commercial Utopia presentations bundle.
\setlength{\pdfpagewidth}{11in}
\setlength{\pdfpageheight}{8.5in}
should give you a landscape slide (on screen) in pdflatex. You may
need to specify in AcroRead's print dialog you want the page
rotated to print properly.
There are a few open source presentation programs that are not LaTeX based. They are more like using PowerPoint. OpenOffice includes a presentation program. GNOME Office includes Agnubis, but it is still work in progress.
Another simple but powerful possibility. The ReportLab Toolkit includes PythonPoint for presentation creation. It is amazingly powerful, but a bit complex. To make things much easier, you can very simply generate PythonPoint documents from restructured text (reST). Once you have the software installed, this is about as easy as it can get!
S5 [2] can simply create beautiful browser based presentations. Instead of editing HTML you can use reST and convert the text file to a slide show with rst2s5. Also see S5 Reloaded and AJAX-S.
Falling short of a presentation package but still potentially useful for report writing is PyReport, which compiles a Python script and its (textual and graphical) output into a PDF file.
Last but not least: for incredible animated presentations, the technically inclined may consider Slithy.
Often it is helpful to be able to annotate a page during presentation. Windows users can use Zoomit. (Proprietary but free.)
One place where a good GUI is nice is for the visual formatting of tables. Here a nice GUI can provide a real advantage over markup. This need not mean abadoning LaTeX.
LyX supports table editing. (You can also copy table data from spreadsheets to LyX.)
(Commercial options include Scientific Notebook and possibly LaTable.) When the structure and content of the tables is the dominant consideration, however, LaTeX tables can often be easily and automatically generated. For example, many econometric packages will produce LaTeX formatted tables of results.
If your chosen econometric package does not support LaTeX table generation, you still have a variety of options.
There are extensive citation management resources.
\usepackage[dviwindo,backref]{hyperref}
If you are new to NetLogo, I have written a basic overview.
Here are some useful NetLogo books:
Swarm is a famous and flexible platform for multi-agent simulation projects, originally written at the Santa Fe Institute in the mid-1990s. (Language: Objective-C or Java; License: GPL) Although the platform is extremely capable, use of Swarm seems to be declining, and there appears to be little ongoing development. The Java Agent-based Simulation (JAS) library attempts to reimplement Swarm in pure Java.
Evolving Objects is "is a templates-based, ANSI-C++ compliant evolutionary computation library". (License: LGPL)
Most scientific programming is done in very high level languages. Two factors to consider in making a language choice are what software in your area is already written in the language, and how much original coding you plan to do.
Julia is probably the right choice for those looking for an alternative to Python and R.
A key reason why Python rocks for research is the NumPy array multi-dimensional library (License: BSD). NumPy is a very high quality library, and it has become my favorite approach to matrix (and multi-dimesional array) programming. This Python package is written as a C extension, so it is very fast. This means you can have full access to the ease of use and to the object-oriented power of the Python programming language while doing serious scientific programming.
Additionally, an extensive scientific computing environment is provided by SciPy. (License: BSD; Dependency: NumPy.) For publication quality graphics, add the excellent graphics package Matplotlib. (License: BSD; Dependency: NumPy.)
Most Python users will be best served by a Python distribution that packages together everything they need as a single install. For example: NumPy, SciPy, and Matplotlib (as well as much more) are included in the following distributions.
Many scientific programming applications can be found on the SciPy Topical Software Wiki. ScientificPython offers some useful scientific and visualization modules. The are also some additional probability and statistics utilities available. Pycoin provides a Python interface to some of the COIN operations research packages. Additional functionality can be obtained from the NAGpy interface to the NAG Fortran library or the PY-ML interface to Mathematica. Python also has modules for interacting with gnuplot and R. You may also want to see the Python math tools at the Vaults of Parnassus. If you need to build GUIs as part of your Python project, consider the Boa Constructor. Additional Python stuff you might find useful for scientific computing: PAIDA (a Python implementation of the AIDA abstract interface for data analysis), and the resources listed at StatPy, and possibly various AI resources. Modular toolkit for Data Processing (MDP) is a Python data processing library implementing Principal Component Analysis (PCA), Independent Component Analysis (ICA), Slow Feature Analysis (SFA), and Growing Neural Gas (GNG). Finally, it is worth poking around CheeseShop.
Being able to write code in one matrix programming language and run it in another environment can be useful. Examples include Treiber's pym Python program for converting MATLAB to Python, Cameron Rookley's gtoml Perl program for converting Gauss code to Matlab, and Christopher Stawarz's i2py script for coverting IDL to Python. If you want to create a translator, consider PLY (a Python implementation of lex and yacc).
Java does not seem to have any eqivalent of Python’s SciPy package for scientific programming. Pieces of the functionality can be obtained as follows. (But stick with SciPy if you can.)
I mention only a couple important items from the endless possibilities.
Popular commercial options include Mathematica, MATLAB, GAUSS, and IDL.
For econometrics, the biggest collections of extant code are in MATLAB and GAUSS,
although Python and Julia code have perhaps caught up.
Of these two, Gauss has the advantage of real array broadcasting,
while MATLAB relies on the relatively clumsy repmat and bsxfun for broadcasting.
Recent versions of Matlab supports classes and GAUSS has structs,
but neither language is truly object oriented,
and both often copy array data when it is unnecessary (instead of referencing it).
If you are going to write a lot of your own code,
avoding needless copies can be critical.
You can get this very nicely with the NumPy Python library.
Interactive Data Language (IDL) is popular among astronomers and others doing intensive image processing. You can run most IDL programs using the GNU Data Language (GDL). You can convert your IDL programs to Python with i2py.
If you are forced to work in Matlab you can still have access to Python via pymex, which "embeds a Python interpreter into Matlab".
Among matrix-oriented programming languages, GAUSS was once the first choice among U.S. econometricians. (Python and Julia now appear to be taking over.) It is known for speed and ease of use. Its collection of built-in statistical procedures is large but not as extensive as specialty languages such as S-Plus/Gnu-S, although this can be offset by its speed and by available free and commercial packages (such as GAUSSX).
Jens Peter Steffen offers a detailed comparison of GAUSS with MatLab and Ox.
This is an extremely short selection of possibly useful free programming languages.
If your emphasis is computational, you will want to visit NIST's Guide to Available Mathematical Software and the Netlib repository. Also, see Jack and Hatem's list of free linear algebra software.
I use Python whenever possible, which is almost always. (The main exception is when execution speed is at an absolute premium in computationally intensive projects.) Traditionally thought of as a scripting language, Python has evolved into an extremely powerful very high level general purpose programming language. It is still useful (indeed, a first choice) for scripting. Use it to find out why. In addition, Python has excellent support for scientific computing. The online documentation is excellent. If you lack programming experience, try Python for the Absolute Beginner and Byte of Python before trying the Python tutorial. Also, if you prefer video tutorials, Microsoft has assembled a multipart video introduction.
One other note: although it is an interpreted language, you can compile Python. Interesting graphics and numerical libraries have been developed for Python.
L3 is a large subset of Python along with a GUI interface that provides a "worksheet" like environment. (License: BSD.) Easy and powerful.
Hackable Games in C: NetHack, derived from the venerable Hack game. Read about in in IEEE Spectrum. See the GitHub repository if you want to hack it.
For scripting,
the most popular choices currently seem to be Python,
Perl,
and Ruby.
There have also been some interesting applications of the embeddable S-Lang interpreter and programmers library.
Lua is especially small and portable,
and is used in the up and coming LuaTeX extension of pdfTeX.
(See my LuaTeX hints.)
It is also worth mention that Jython implements Python as Java,
allowing the ease of Python scripting to be combined with the power of Java.
(As a very cool plus,
Vim supports scripting with Perl, Python, and Ruby.)
More Scripting:
Active Perl is available for many platforms and is getting excellent reviews.
Python is growing in popularity,
as is Ruby
(which finally has a Windows installer).
Haskell is worth considering. Graham Hutton complete Youtube course on Haskell. The massiv package supports multidimensional array operations.
We have alread discussed C above. Certainly C++ and Fortran have to be mentioned. The GNU Compiler Collection (GCC) includes front ends and libraries for these (as well as Objective-C, Java, and Ada).
Getting started with Java:
Download the Java Development Kit (JDK),
which includes the Java Runtime Environment (JRE).
On Windows, you install this by running the downloaded executable.
Then download Eclipse,
which is an IDE that is popular for Java development.
You “install” Eclipse simply by decompressing it.
(E.g., on Windows, just unzip it.
It does not make any registry changes.)
Eclipse comes with tutorials.
For a simple overview of installation,
consider this online video.
For very simple introductions to Java (suitable for those without programming skills), try David Eck’s free book and/or Java Programming for Kids, Parents, and Grandparents.
The F
programming language is a carefully selected
subset of the most useful Fortran 95.
You can compile F with the g95 compiler:
g95 -std=F my_program.f95,
available for free from G95.
FreePascal includes very substantial Delphi compatibility. (License: LGPL and GPL)
Ada, which is a preferred languge for the security conscious.
Eiffel supports desing by contract. There is now open source licensing of Eiffel Studio. (License: GPL)
Scheme.
MzScheme is a free and open source implementation for Linux, Mac, and Windows.
(As a very cool plus,
as of version 7 Vim supports MzScheme scripting.)
Scheme, and R. Kent Dybvig's The Scheme Programming Language, Second Edition is now online.
Sather-K is Eiffel based but adds innovations (and is free).
Visual Eiffel Lite is free. Documentation is in Eiffel: Object Oriented Programming
PostScript is discussed under Drawing Languages.
See the free stat software pages by John C. Pezzullo and Gene Shackman . (The latter offers some package comparisons.) Also see the QM&RBC Software page, Spatial Statistical Resources, the Econometrics Laboratory Software Archive (ELSA) at Berkeley, and many others.
Resources in this area are extensive. I will only mention a few idiosyncratically selected items. (For linear and non-linear programming, see also the separate list of solvers.) Do not forget to think about scaling issues. (Here is one discussion of scaling: go to the Aptech web site, click on Resource Library, then click on White Papers, and download Optimization with the Quasi-Newton Method.)
Visual analysis of data remains very useful,
and some innovations in this area are recent.
Classic texts include:
The Visual Display of Quantitative Information by Edward Tufte
and
Visualizing Data by William S. Cleveland.
You can get the data and S scripts for Cleveland's book.
Cleveland (with Becker and Shyu) developed Trellis Graphics™ for data visualization,
which has been implemented for a number of platforms including S-Plus and R.
(The name “Trellis” is trade-marked, so these are often referred to as lattice plots.)
Other resources:
Daniel Carr's page.
For interactive data visualizations, you must consider Vega and Vega-Lite. These visualization tools implement a grammar of graphics and a grammar of interactivity, mostly using declarative specifications in JSON. Vega-Lite provides a convenient, very high level access to this grammar. Many languages now support this, including JavaScript, Python (Altair), Julia (VegaLite.jl), and Haskell (hvega). Voyager 2 is an interactive data exploration tool built on top of Vega. Some languages support this, including JavaScript, Python (Altair), Julia (DataVoyager.jl).
Modrian "is a statistical data-visualization system written in JAVA." Good for categorical data and for maps. Plots are interactive. There is a supporting paper by Martin Theus.
Find a more list of Computer Algebra Systems (CASs) on Wikipedia. This section discusses only a few particularly interesting applications. For most users, commercial CASs will be the right choice, but some interesting FOSS solutions exist.
There are a number of way to provide access to nicely formatted math on the Web. The most widely used is to provide PDF documents that contain nicely formatted math. The best way to produce good looking math in PDF documents is with TeX based software. Acceptable results can sometimes be achieved by converting a word processor document with equation editor objects embedded in it.
The other major alternative is to somehow produce math in a web page.
The stand for this is MathML,
but as late as 2021,
FireFox and Opera remain the only major browsers with good support.
(Be sure to download the STIX fonts.)
Producing MathML by hand is impractical,
so there has been a search for feasible ways to rely on MathML.
One of the most interesting approaches is to use reStructuredText
with the rst2html5 writer.
This approach uses LaTeX syntax to input mathematics.
Possible output formats
include MathML and MathJax.
This is probably the simplest powerful way to put math on the web.
Another interesting approache is ASCIIMathML,
which simply includes LaTeX-like math in a normal HTML file.
(A clever javascript renders the math.)
Mozilla based browsers such as FireFox have excellent native MathML support,
so they work well with ASCIIMathML.
(But you need to install the math fonts.
See the Wikipedia MathML article
for a summary of support by other browsers.
One other alternative is to convert LaTeX documents directly to XHTML+MathML documents. This can be done with LaTeXML, TeX4ht, or with the closely related TeX4moz. TtM looks like a similar effort.
If you want to test your browser for standards compliance, start with the Acid 3 tests. You might also want to look at handling of MathMl, SVG graphics, or even a very simple XHTML. (Be sure to look at the tests where the document is served as XML.)
I like three browsers: FireFox, Opera, and Chrome. Of these, I give FireFox the nod for being both open source and mature, but Chrome is also open source and is considered faster. Microsoft Edge is slowly becoming usable; it is certainly much more standards compliant than the old Internet Explorer. Try them all on the Acid 3 tests. An interesting "new kid on the block" is uzbl, which emphasizes speedy browsing but is highly extensible.
See the Opera hints below. See the FireFox hints below.
The ePub3 standard introduced MathML support. Quite a few ePub readers support MathJax.
Request:
Please follow basic email etiquette [1|2],
by quoting selectively and only as necessary to provide context for your reply.
When appropriate use only ASCII text for your email.
Finally, if you must use Outlook,
do your interlocutors the courtesy of turning off TNEF attachments!
Get a decent email client. Since there are so many good options, choose an email client that has a multi-platform presence and, most importantly, lets you use your own editor.
Suggestion:
Using your own editor will repay you handsomely, saving time and frustration.
Here are examples of things any good editor will allow you to do simply.
Preprocess your email to compensate for bad practices by the sender,
delete or reformat blocks of text (e.g., a line, a sentence, or a paragraph) without reaching for a mouse,
adjust capitalization (e.g., change caps to lower case or vice versa, or convert to first letter caps),
and quickly insert text from any other file (i.e., read it in, so you do not need to cut and paste).
One could propose that every application in which you need to edit text should include such facilities.
E.g., there are many email clients with built in editors that have many such facilities.
But then each time you change applications you will have to learn a new editing environment.
Instead, whenver possible, you should use applications that provide support for an external text editor.
By sticking with your preferred editor you do not have to learn a new editing environment and scripting language each time you switch applications.
That of course is the point of modularity.
See the Thunderbird hints and Alpine hints below.
Another cross platform solution is now The Mutt Email Client—long popular among Unix users, and available to Windows users. (You can also get mutt along with fetchmail, and sSMTP in a unix-mail package for Windows, but you'll have to install Perl and Cygwin.) An email client that is not free but gets great reviews is The Bat, and but it seems to not yet support external editors. Another popular cross-platform email client is Mulberry, and it is available on many platforms. (License: Apache 2.0) One last note on email: if you use Vim as your editor, you might consider using vimail, a simple mail reader for Vim.If you are in an environment that forces you to use Outlook, you can at least write your email with a decent editor.
POPFile sorts your email and filters spam. It is free and cross platform.
What are some good document formats, and how can one transform between them?
For writing math, nothing beats LaTeX. However, you can use reStructuredText with writers that support math to produce both LaTeX and XHTML MathML.
Writing HTML is easy: learn the basics. Conversion to other formats is possible. Most word processors will read HTML and automatically convert to their own format. You can also convert HTML to PostScript (if you have access to Perl).
After learning a few HTML basics, you can learn much more from the materials at the World Wide Web Consortium W3C). Most of the time you should be able to use XHTML Basic and Cascading Style Sheets (CSS). Here is some CSS Background, and the wonderful chapter 1 of Cascading Style Sheets 2.0 Programmer's Reference by Eric A. Meyer. For more information and links, go to the HyperText Markup Language (HTML) Homepage. Some authors may like to see the ISO standard as well.
All modern Word Processors can export their documents as HTML: unfortunately, they often produce pretty bad HTML. You can get much better results with some of the HTML editors, such as the web development suite offered free by Evrsoft
XHTML is a wonderful HTML related markup which provides for most needs when used with CSS. Unfortunately some browsers do not support it, unless you serve it as HTML. (That is the trick used in the present document.) Some authors discourage this practice, but it should work fine, as long as you validate your documents.
XHTML: Introduction to XHTML, with eXamples" helpful | XHTML validation. With XML/XHTML (handled by an XML processor) you will be able to define your own entities in the doctype declaration. E.g., you could define <!ENTITY alpha "&#xxxx;" > (where xxxx = the numeric reference) to get around remembering numeric references. The XHTML+MathML document type looks particularly useful, and it is already used to provide math support in reStructuredText.
Mathematical Markup Language (MathML) is now a wonderful reality. FireFox and other Mozilla based browsers offer native support. So does E-Lite: ICESoft's lightweight Java web browser offers full, native MathML implementation. Internet Explorer browsers currently support it only via plug-ins. Public discussion on features for math in HTML contexts takes place on www-math@w3.org. To subscribe send a message to www-math-request@w3.org with "subscribe" as the content of the subject. There is a public archive. Also, consider WebEQ for Web math.
XML:
Perhaps the most interesting current XML development for non-programmers is
OpenDocument Format (ODF).
The is a promising open document exchange standard.
The OpenDocument Format is an open, fully-documented, royalty-free standard for document exchange.
It is supported by OpenOffice
and the closely related LibreOffice.
There is also a plug-in converter for MS Word (TM).
By the way, docutils can write directly to ODF:
see Dave Kuhlman's work (currently in the docutils sandbox).
What is that winmail.dat attachment you just got in your email? It is a proprietary Microsoft format that bundles together multiple attachments in a way that most email clients cannot unbundle. How obnoxious! So, what can you do about it?
One possibility: ask your interlocutor for the simple courtesy of turning off TNEF attachments! (Instructions are the bottom of the article.)
If you are unfortunate enough to receive a TNEF attachment from someone who cannot figure out how to configure their email client to send attachments that everyone can read, you may have to unpack the TNEF attachment yourself. The only cross platform solutions I know of are JTNEF and pytnef (License: LGPL). At this point, pytnef relies on the tnef command-line utility that is I think only available for unices (although it looks like it could be compiled for Windows).
If you need to produce email in a scripting environment, Python offers all the basics in the smtplib module. You can even add MIME attachments with reasonable ease.
Cross platform: Pan (GPL) or slrn or Gnus.
Free for Windows: Xnews and Forté Free Agent.
First let me offer an apparently outrageous suggestion: individual users can often substitute good habits for AV software, and even if you install AV software these good habits are crucial defenses. That said, whatever software you choose, you may wish to test it on the EICAR test file.
TO AVOID SENDING AND RECEIVING VIRUSES:
SENDING
i. use plain text to send and receive
information whenever possible.
All word processors and spread sheets
can save as .txt.
There is seldom a real need for using
anything except plain text for email.
ii. When plain text is not possible
(because special formatting is required)
you can almost always use HTML.
All modern word processors
and spread sheets can save as HTML.
This is almost risk free, but to be
really safe you can turn off Java and
JavaScript in your browser preferences.
iii. When page formatting is crucial,
which it almost never is,
use Adobe's PDF format.
You may need a PDF "printer driver" to do this
(such as Adobe Acrobat, Jaws, or some free alternatives such as PDFCreator).
RECEIVING:
i. Many viruses are received as part of email attachments.
Other malware (known as worms) will email
themselves from an infected computer to many others.
So the best practice would be to not open any email attachments.
Unforunately, sometimes we need to open attachments.
ii. If you must open an attachment, view the file extension.
The default in Windows is to turn off your ability to do this,
which means you can be tricked into thinking a dangerous
Visual Basic script filename.jpg.vbs
is really a safe .jpg file.
You can fix by following this windows hint.
For Windows users,
especially dangerous extensions are .exe, .com, .pif, .vbs.
Also dangerous are .doc and .xls.
(Comment: dangerous file types can be included as ``attachments'' to .pdf files as well,
which means you should not open such attachments with Adobe Acrobat either.)
iii. When possible, avoid the use of susceptible
proprietary file formats (such as Word or Excel).
Related to this, avoid the use of Outlook unless you know
how to turn off all its dangerous features.
And if you simply must open a document
in a MS format (e.g., because that's what
a work colleague sent you), learn how to
turn off the availability of macros.
You should also be cautious with any other
format that supports macros, which basically
means you should avoid exchanging information
in word processor formats (see i. and ii. above).
iii. Never run a program file unless you
know it came from an impeccable source.
(For Windows users,
this includes file extenstions .exe, .com, .pif, .bat, and .vbs.)
iv. Windows users should disable autoplay,
since it allows arbitary executable to run automatically on your computer.
Additional Comment: Many new viruses are not detected by anti-virus software. Safe practices are necessary. Furthermore, if the recipient wishes to read what you send in their word processor, all modern word processors also import plain text (very safe), HTML (pretty safe), and PDF (usually safe) very simply.
Since this is not a core interest of mine, I list only a couple resources.
Attention Vista users: your operating system came without the Windows Media Player plugin. Fortunately, mozillaZine explains how to install the WMP plugin.
The modeling languages most widely used by economists are commercial: GAMS (with online documenation) and GEMPACK (with online documentation). Economists use these primarily for CGE modeling. (See these helpful CGE exercises with solutions.)
Other alternatives are listed at http://plato.la.asu.edu/topics/tools.html. A few are listed below.
Naturally, many commercial modeling languages come with a variety of solvers. Here are some other solvers.
For Python, PyScripter has an excellent reputation. But it has limited support for threading. (Windows only; license unknown.)
For Java, consider the Eclipse IDE. (Eclipse is a "general purpose" IDE.) Python users can also use Eclipse via the Pydev plugin.
For C and C++, consider the Agide IDE. You can use Vim as your editor! On Linux/KDE, you can alternatively use KDevelop, which also support Vim as an editor.
VimMate is a light GTK-based IDE that embeds the Vim editor.
For Python:
- the general purpose Leo IDE includes a unique and powerful outlining facility (Language: Python; License: MIT)
- the relatively lightweight Eric IDE gets very good reviews. (Language: Python; License: GPLv3)
- Jedi helps turn VIM into a Python IDE by adding autocomplete.
For Android, Google has released Android Studio in 2014.
ECMAScript is a standaradization of JavaScript, a dynamic, interpreted programming language whose origins are in client-side scripting in the browser. Most browsers support ECMAScript, and client-side scripting has been used to create economic applications. [#paine.stark-2004-cheer] These features made it a widely used language. Growth in language features, including support for functional programming, have made it a rather interesting language as well.
If you are new to JavaScript, I have written a basic introduction.
Python is an obvious choice for server side scripting. You might consider Xitami as a cross platform server solution, or simply to test your scripts.
The obvious language independent FOSS choices currently (2005) seem to be TK and wxWindows, with TK being simpler, more mature, and a bit slower and wxWindows being faster and more powerful but less stable. Of course Java provides a language dependent solution.
Python comes with the Tkinter module, which provides a Python interface to the TK graphical user interface (GUI). Tkinter documentation is spread around. For a very brief introduction, you can try Alan Gauld's tutorial. If you use Tkinter, consider using Python megawidgits (Pmw). (You can also easily add graphing extensions with BLT Toolkit. You may want to also get Mark Rivers's enhancements.
tk_happy allows very fast, menu guided tkinter GUI creation.
Until recently curses has been unavailable for Windows. Wcurses is a curses module for Python that works under Windows. There may also be some profit in considering curses applications on INTERIX.
wxWidgets offers cross-platform API (C++ library) for writing GUI applications. wxPython is a Python extension module that wraps wxWidgets. Dabo is a cross-platform application development framework, written in Python atop the wxPython GUI toolkit. (Dabo capabilities resemble Visual Basic or Visual FoxPro, but it's cross platform.)
There is some useful background on the PyMol wiki.
Encapsulated PostScript (EPS) remains a very important standard for graphics inclusion in journals and books. It therefore pays to learn a drawing application or language that allows you to generate encapsulated PostScript (EPS). You can convert EPS to PNG or SVG. (See below.)
In many applications, EPS) is being replaced by Adobe's Portable Document Format (PDF). PDF) and page indepdence and support for new features, but loses the powerful PostScript drawing language.
Currently the best choice for web graphics. (GIF and JPEG suffer patent problems, although these patents are finally expiring.) Before placing graphics on the Web, visit the Portable Network Graphics (PNG) homesite.
MNG and APNG
The animated version of PNG is called
MNG, for Multiple-image Network Graphics.
It supports looping (including nested loops),
clipping, deltas, and other features, plus
everything PNG supports-- including alpha
transparency.
Unfortunately implementations have been lagging.
However FireFox supports the animated PNG (APNG) format,
which is less complete but quite capable.
With all the point and click drawing applications currently available, why bother with a drawing language? The answers are generally precision, speed, and fun, often in that order. This section concentrates on 2-D drawing, with emphasis on the PostScript drawing language, while mentioning a few other interesting possibilities.
The so-called Grammar of Graphics has garnered attention because of its power and expressiveness. PlotNine is an implementation for Python. ggplot2 is an implementation for R. It is supported by two popular books: R Graphics Cookbook and ggplot2: Elegant Graphics for Data Analysis.
PostScript is a device independent page description language. If you want to precisely render a vector graphic, you may well find it convenient to draw it yourself in PostScript. I offer an introduction to PostScript Drawing for the Sciences. (Still under revision; comments welcome!) See some PostScript hints below.
GhostScript and GhostView let you view and print PostScript files.
Related: PLisp takes the basic structure of Common Lisp and uses it as an alternative representation of a PostScript program.
While the basic graphic operators of PostScript are retained, Lisp-like function definition, control flow,
and variable binding become available.
Macros, defined constants, and function libraries have also been provided.
The PLisp compiler generates PostScript code which is guaranteed not to contain errors in stack usage.
SVG
This popular format for web graphics can be useful when you need vector graphics and/or animation.
It was supposed to be possible to embed SVG in PDF,
and for a while Adboe worked actively on an SVG viewer (which is still available).
This seems to have stalled,
and indeed further development of the SVG standard unfortunately seems to have stalled.
For extensive SVG manipulation and generation tools,
see Batik (language: Java).
An increasing number of graphics languages offer an SVG backend.
Scalable Vector Graphics is a royalty free vector graphics standard for the Web,
and it is a powerful animation format.
(If you have simple, static graphics to produce, you might write them in PostScript and then translate using pstoedit.)
One of many nice things about SVG is that it can be embedded into PDF,
animations and all!
(See this very cool example.
Here is a perl script.
Here is a Java converter.
Here is a Python script.
(If you can find Leonard Rosenthol's Acrobat plug-in, let me know.
It seems to have disappeared.)
Another nice thing is that you can move image generation to the client side.
Browser support for SVG is improving but advanced features may be missing. Check your target browser. (You may find some useful information on Wikipedia.)
Curren Kelleher offers an extensive tutorial on data visualization using SVG (via D3). The tutorial uses VizHub (MIT license).
3D drawing: For simplicity and power, it will be hard to beat Sketch. (Langauge: C++ (binaries available); License GPL) Particularly helpful to LaTeX users is its support of tikz. (See Kjell Magne Fauske's nice introduction to Sketch, which includes full examples.)
Try the Persistence of Vision Ray Tracer (POV-Ray). If you need high performance real-time 3D, consider Irrlicht. If you are an OpenGL user, superficie may be useful, although development appears quiescent. (License: GPL)
3D plotting: For nice, accurate surface plots, there is probably no simpler solution than gnuplot. The SymPy plotting module is very interesting. QwtPlot3D appears very powerful, and PyQwt3D provides bindings for Python users who wish to produce scientific 3D plots. (Now supports NumPy arrays.). Other possibilities include Visual Python and Python Computer Graphics Kit. Also, OpenDX looks like a very powerful open source implementation of the IBM Data Explorer (DX), and there is also a Python binding. VisIt looks to be an extremely powerful way to produce 3D animations.
The Visualization Toolkit (VTK)
"software system for 3D computer graphics, image processing, and visualization"
consisting of "a C++ class library, and several interpreted interface layers including Tcl/Tk, Java, and Python."
(License: BSD-like, but containing some patented technology.)
Hint: Find yourself a good interface to VTK.
VTK provides wrappers for Python, Tcl, and Java, but the wrappers resemble VTK more than the supported language.
Python users may like MayaVi,
but this still requires VTK knowledge.
For true ease of use, consider Enthought's mlab Python framework for access and manipulation of MayaVi2 objects.
(Also note that PyVTK can write VTK files from NumPy data.
TVTK provides a powerful and "Pythonic" interface to the Visualization Toolkit.
With some feature resemblance to MayaVi,
the possibly more powerful Paraview is written in Tcl/Tk rather than Python.
More 3D plotting solutions:
HippoDraw produces interactive graphics for data analysis. Hippodraw includes Python bindings and a Python extension module.
Simon Yuill offers a nice animation tutorial (with a Python emphasis).
Pms.Blt provides a Python interface to the BLT Graph plotting library.
mencoder mf://*.png -mf type=png:w=600:h=500:fps=10 -ovc lavc -lavcopts vcodec=wmv2 -oac copy -o animation.avi
or like this
mencoder 'mf://*.png' -mf type=png:fps=12 -ovc lavc -lavcopts vcodec=mpeg4 -o animation.avi
.)
If you have mpeg2encode,
this can also be done with ImageMagick,
try convert -delay 20 c:/myimages/img*.png c:/myvideo/imgs.mpg.
winget install "FFmpeg (Essentials Build)".
Blender supports powerful 3D drawing and animation, which can be assisted by Python scripts.
HarddiskOgg can record analog audio "directly to Ogg Vorbis/MP3/Wave with on-the-fly normalization". (License: GPL)
HTML % includes a VIDEO element, supported by FireFox 3.5+ and Chrome 3.0+. (See Video For Everybody for usage hints.) This enables native OGG/Theora video playback in Firefox 3.5! For conversion of video formats, you might try HandBrake. (License: GPL)
See the SciPy TopicalSoftware Wiki for a more complete list of options.
pyplot module provides quick-and-easy MATLAB-like graphing functionality.
For finer grained control, you may prefer Matplotlib's object-oriented API.
Matplotlib can save to a variety of formats,
including SVG (which you can then manipulate with Inkscape).
See the Matplotlib hints below.
A cross-platform standard for graphics is Encapsulated PostScript (EPS). EPS is sometimes the expected graphics format by publishers when you submit drawings for a book or journal article, although PDF is becoming more universal. A common file conversion problem remains taking a file produced in a different format and converting it to EPS.
PNG is the current standard for web images. If you have produced an image for the web in a different format, see the LibPNG list of converters. There are many options listed there, but if your image is PostScript or PDF, consider GhostScript.
GhostScript can convert many file formats, including PS to PDF. You can combine GhostScript with the Redirection Port Monitor (RedMon) to produce PDF files (see helpful directions). FreeDist is a Visual Basic program can automate this process for Windows users. FreePDF seems to offer similar functionality to FreeDist. PDF995 is another free (gratis) but not open source alternative.
If you need to convert entire HTML documents to PDF, the easiest way to is to open it in a browser that supports conversion and simply save as PDF. (Chrome supports this as a Print destination.) There are also stand-alone converters. Try HTMLDOC. Perl users can try html2ps and then use GhostScript. If your document is well formed XML, CSSToXSLFO offers very interesting possibility. It converts an XML document, together with a CSS2 style sheet, into an XSL-FO document. The document "can then be converted into PDF, PostScript, etc. with an XSL-FO-processor", which is included. (License: Public Domain)
There is an almost limitless variety of high quality commercial drawing applications with pricing roughly linked to functionality. Rather than attempt to list or compare those, I will focus on a few cross-platform alternatives.
Python users have a number of fine alternatives. For serious 2-D drawing, I like PyX very much. PyX also has some support for 3D graphics. (See my PyX hints below.)
While PyX is great for data-based graphs, I often find myself using Matplotlib for these. (Matplotlib is a wonderful 2-D plotting package which also supports powerful interactive graphics.) PyX and Matplotlib support the generation of EPS and PDF plots with LaTeX text. (Additionally, Matplotlib provides an impressive mathtext capability independent of LaTeX.)
Matplotlib and Pyx currently take different approaches to math text. PyX uses LaTeX to typeset the math, so you must have a working LaTeX installation. Pyx generates PostScript, PDF, and SVG graphics with this approach. Matplotlib uses its own parser and layout engine to typeset math text with Type 1 computer modern fonts. You do not get the full power of TeX typesetting, but you can produce outputs in a variety of formats (including EPS, PNG, and PGF). Here are some details.
Matplotlib supports spark graphs (sparklines) via the sparkplot module. (Sparklines can also be produced with PIL using the Sparkline Generator Web Application.)
Other possibilities:
Asymptote is a powerful and popular language for drawing 2D and 3D graphs.
The syntax resembles Metapost.
Support LaTeX typesetting of labels.
Asymptote allows easy generation of animations (animated GIFs and MPEGs)!
(License: GPL 3.0)
Kiva
"is a multi-platform Display PDF 2D drawing engine that supports multiple output backends, including wxWidgets on Windows, GTK, and Carbon, a variety of raster image formats, PDF, and Postscript."
(Requires ReportLab for PDF output.)
PyChart is excellent for quick, simple, flexible charts.
DISLIN is free for non-commercial use and is available as Python extensions.
VUESZ is a PyQT based scientific plotting package.
geocat module that relies on the matplotlib plotting library.
Eukleides is an amazing Euclidean drawing language that can save your drawings as pstricks or in EPS format. It is designed to be very natural for geometric thinking. (License: GPL)
Graphviz
is an open source package with special abilities in drawing directed (via dot) and undirected (via neato) graphs.
The focus is on Graph Visualization.
pydot provides a Python interface to dot.
yED is a
"powerful graph editor that is written entirely in the Java programming language.
It can be used to quickly and effectively generate drawings and to apply automatic layouts to a range of different diagrams and networks."
Artdraw may prove promising: it is a PostScript-like syntax that adds alpha blending.
If you are considering PostScript graphics on the way to PDF documents, is worth noting that ReportLab allows direct creation of PDF graphics and reports.
Before exploring other possibilities,
try gnuplot.
It meets most non-interactive data driven or function driven plotting needs.
It is free and open source, multiplatform, very fast, and very powerful.
(There is a Mac version,
and here are some instructions for installing under OSX.)
You can download official release.
New users may profit from Langlais's "short course"
or
the IBM tutorial.
Features: In addition to plotting 2-D and 3-D graphs of arbitrary functions,
gnuplot can also quickly plot large data sets.
It has a powerful yet intuitive scripting language.
GNUPLOT can provide output in many different formats,
including the LaTeX picture environment and EPS.
The LaTeX and epslatex output "terminals" allow for math symbols etc.
(After LaTeXing the resulting .tex file, the .dvi file can be converted to PostScript using dvips -o.
The Ghostview previewer can then be used to convert the PostScript file,
fonts and all,
to EPS.)
All told this is quite a powerful plotting tool with extremely flexible output.
New users should read Introduction to GnuPlot,
but skip the institution specific material on starting gnuplot and head straight for the "Simple Example".
See the gnuplot hints below.
Unified Modeling Language (UML) may not be truly unified—it is clumsy at best for functional programming—but it still offers a remarkably useful collection of chart types for modeling and simulation. Here is a tiny selection of available resources, plus a few (somwhat) related flowchart resources. For more options, see Wikipedia's list.
Papyrus seems the most complete option. However, it is a complete modeling environment, and it is accordingly heavy weight. The remaining options are primarily for drawing key UML diagrams.
Here are a couple older flowchart tools.
Scripted Image Manipulation with Python
Edward Tufte offers an online discussion of sparklines.
sparkplot.py creates sparkplots using Python and Matplotlib.
import Image, ImageDraw
import StringIO
import urllib
def plot_sparkline(results):
"""Returns a sparkline image as a data: URI.
The source data is a list of values between
0 and 100. Values greater than 95
are displayed in red, otherwise they are displayed
in green"""
im = Image.new("RGB", (len(results)*2, 15), 'white')
draw = ImageDraw.Draw(im)
for (r, i) in zip(results, range(0, len(results)*2, 2)):
color = (r > 50) and "red" or "gray"
draw.line((i, im.size[1]-r/10-4, i, (im.size[1]-r/10)), fill=color)
del draw
f = StringIO.StringIO()
im.save(f, "PNG")
return 'data:image/png,' + urllib.quote(f.getvalue())
if __name__ == "__main__":
import random
html = """
<html>
<body>
<p>Does my sparkline
<img src="%s">
fit in a nice paragraph of text?
</p>
</body>
</html>"""
print html % plot_sparkline([random.randint(0, 100) for i in range(30)])
If you produce documents in LaTeX, you would naturally like the fonts and equations in your graphics to match those in your text. There are a variety of solutions. Nowadays the simplest answer by far is to learn to use TikZ.
For great results even in complex drawings, try the PyX graphics package for Python. This will require an initial investment of experimentation and learning, but the results (as EPS or PDF) are fabulous.
\usepackage[pdftex]{graphicx}\usepackage{epstopdf}.
--shell-escape command line switch.
\includegraphics{filename.eps} as usual to include your EPS graphic,
and pdflatex will conver them to pdf format on the fly.
(Be sure to return the extension! Otherwise you will get conversion the first time,
and use of the old PDF after that.)
-dEmbedAllFonts=true.
See this document
for more discussion.
Laptop (and cell phone) theft is a constant problem. The Prey Project offers a multiplatform, FOSS theft recovery tool. It uses Google geo location and a web camera if available, and is available for the Android operating system. (A related commercial product is Undercover by Orbicule, which has additional functionality and is very user friendly.) Here are some other possible steps to take. You can run autossh as a service that creates a link between your home router and laptop. Then whenever your laptop has internet, you can ssh into it from home. You can also install a keylogger and schedule (e.g., with launchd or cron) to say sftp the log every day and then restart the log file. (Personally I would only use open source keyloggers, such as PyKeylogger, but these tend to be less focused on hiding themselves.) Probably better than any of this is vigilant use of a laptop cable lock.
Quantum GIS is an extensible geographic information system. (License: GPL)
Planning an open access journal (business models, tips, etc.). Software to create and manage online open access repositories for articles: Eprints (from Southampton University, perl); DSpace (from MIT, java); CDSWare (from CERN, python); FEDORA (from Cornell and U. of Virginia, java). Also: Copernicus is a company specializing in the publication of open access journals.
RaspberryPi now has an official manual, which includes introductions to programming languages (including Python and Java).
Windows keyboards are a painful design for touch typists. Consider Happy Hacker Lite II, which unfortunately is a bit pricey. Short of that, at least remap the prominent caps-lock key to something useful: [1 | 2 ]
Touch typing will speed your writing and programming immensely. I know of two free cross-platform solutions. Tux Typing is targeted to kids but is generally useful. Vim users can use the TTCoach script.
Cross-platform PIMs Chandler, which is very promising. (License: Apache. Language: Python.) Also look at Aethera. In the meantime, the Palm Desktop is free, available on Windows and Mac, and quite functional (although it does not use the iCalendar standard).
Citadel is a promising groupware application.
There are lots of good calendars out there. But if you are "always connected" I have not been able to beat Google Calendar. (I have not tried to use it offline.) Note that you can add Google Calendar to your FireFox sidebar. Comment: if you ever see a corrupted screen when viewing Google Calendar, try clearing your browser's cache and cookies.
Generally it seems a good idea that your calendar support the iCalendar standard. (This means you can easily move it between applications and across platforms.) It also means you can easily write scripts that interact with your calendar. For example, you can use the iCalendar. (License: LGPL; Language: Python.) An alternative is the Python-PDI library for personal data interchange, which also handles the vCard format. (License: GPL; Language: Python.) Aside: GCALDaemon offers synchronization between Google Calendar and iCalendar compatible calendar applications. (Language: Java; license: APACHE version 2.0) (Gina Trapani offers some installation hints.)
I like Sunbird a lot (it is the Mozilla offering). Going under the name of Lightening is a calendar extension for Thunderbird that shares the Sunbird code base. For an interesting and "light" calendar, you can try Remind. (Linux Journal has a review.)
The address book situation is a scandal. Applications have not settled on a standard for data sharing and do not even adequately document their own formats. The best documented formats, VCARD and LDIF, are not great. For such a widespread need, this is a rather astonishing situation.
Thunderbird comes with a pretty good address book tool. Unfortunately, it saves address books in Mork (.mab) format, and if you back these up it cannot import them! Fortunately, there is a Mork parser library (Language: Python) Thunderbird 3 was supposed to replace the horrible Mork format with MozStorage, thereby adopting the storage format now used in FireFox. But the current situation is still bad. However, there is an address book extension that improves things.
Most PIMs include address book functionality. Most email clients do as well. You can also get such (database) functionality in stand-alone applications. On Linux you can use Gaby or GTK-Agenda. There is also the Contacts application.
See Wikipedia for a comparison of project management software. Here is another comparison.
Here is one that is not yet listed there: ProjeLead (License: GPL) Probably Trac is best known in the open source community.
Open Financial Exchange (OFX) "is a unified specification for the electronic exchange of financial data". The format is an open standard. It is possible to convert CSV to OFX.
Advice on office suites: do not use them for research. Word processors may slow down your writing; use a good text editor. Spreadsheets are a risky way to handle data [ 1 | 2 | 3 | 4 | 5 | 6 ]: use a simple matrix programming language instead. The virtues of existing presentation packages do not outweigh their weaknesses, especially if you need to present mathematical formulae.
Nevertheless, office suites can be convenient. Here are some basic considerations. FOSS office suites are currently a high-quality multi-platform solution. I recommend either LibreOffice (License: LGPL) or OpenOffice (License: Apache). Of course most of the people you know will be using a commercial office suite, but you will be able to save your files in formats they can read, and as long as they are not using advanced features (e.g., templates, complex macros, or complex spreadsheet formulae) you will be able to open their files as well. Most users do not use such features. FOSS suites meet most personal and business needs.
Since office suites are often expensive,
here are some cheap or free commercial alternatives.
Google Drive
(good intermediate functionality; easy collaboration;
limited advanced functionality)
Word Perfect Office
(home of the famous Word Perfect word processor)
Soft.Maker is a cross-platform commercial suite.
Hancom Office
We exchange documents for three primary reasons: for viewing, for interaction, and for modification. Word processor formats are a popular choice, and the versioning capabilities of modern word processors can be very useful in a collaborative setting. Plain text can often satisfy viewing and modification needs, and there are great version control tools for text formats. When more structure is needed, HTML will usually suffice. When formatting must be tightly controlled or access needs to be controlled, PDF is often a good choice.
If you want even lighter weight than the very nice AbiWord, on Linux you can try Ted. Some Linux users like frames-oriented KWord.
Nowadays it is perfectly feasible to use a decent HTML editor as your word processor. The Amaya browser/authoring tool may be worth considering, although I have found it a bit slow and clunky. (Somewhat offsetting this is its native MathML support.) Another possibility is the web development suite offered free by Evrsoft.
There have been some interesting efforts to provide fairly direct use of PostScript for document markup. A popular one was Graham Freeman's QuickScript, but I can no longer find it on the web. Another alternative, Tinydict, is still around.
It is now perfectly possible to rely on internet applications for word processing. I have had good experiences with Google Docs for basic word processing tasks and online collaboration. Zoho Writer looks even more capable, and has good math support as well. (Including the ability to produce math by inputting LaTeX.)
You will need to decide what format to keep your bibliography in, and what software to use to manage it.
Citation Management:
If you use a commercial word processor you will probably use the commercial EndNote
to manage your reference and format your citations.
However there are some interesting alternatives.
In particular,
LaTeX users have the traditional (and very powerful) BibTeX to manage the citations in their documents.
Even users of commercial word processors can take advantage of BibTeX by using Bibtex4Word.
New users should see the BibTeX user hints.
Now for the design of citation reference and citation style,
they have the even more powerful biblatex
(which greatly simplifies writing new styles).
(Users switching from natbib will want to know about the natbib-compatibility-mode,
which allows use of natbib's commands (e.g., \citep, \citet)
instead of the commands usually provided by biblatex (e.g., \parencite, \textcite).)
Python users can consider the bib4txt component of bibstuff,
which should particularly interest reStructuredText users who need bibliographic support.
(I recommend you get the most recent BibStuff.) (BibStuff License: MIT; Language: Python)
Chris Putnam's Bibutils
"interconverts between various bibliography formats using a common XML intermediate."
(Bibutils License: GPL; Language: C; binaries are available for many platforms.)
Users of Open Office or MS Word can insert references directly in their documents with Bibus.
CiteULike is an interesting "in the cloud" solution for managing references.
The current choice for citation management appears to be Biber+Biblatex. (Other possibilities are discussed below.) The traditional choice for citation management is BibTeX. (See the BibTeX user hints.)
Here is how to transition from
the traditional Bibtex with natbib to the more modern Biber with biblatex.
(If you have not been using natbib,
you can leave out that option.)
After changing your LaTeX document as indicated below
and compiling it with pdflatex as usual,
run biber on the document’s basename
(instead of using Bibtex).
\documentclass{article}
%\usepackage{natbib} %REMOVE natbib package
%ADD biblatex package (with Biber backend)
\usepackage[
backend=biber,
style=authoryear-comp,
natbib=true,
sortlocale=en_US
]{biblatex}
%ADD .bib files (one by one, in preamble)
\addbibresource{mybib.bib}
\begin{document}
Lorem ipsum dolor \dots
\printbibliography %ADD printbibliography command
%REMOVE old bibliography commands
%\bibliographystyle{mystyle}
%\bibliography{mybib}
\end{document}
In the background,
the choice of the biber back end for biblatex
will lead to to the creation of a .bcf file,
which biber operates on.
But you should not need to know about this
unless choose to pass to biber
the full filename (basename + .bcf) instead of just the basename.
In your document,
you will need to keep the natbib option to biblatex
as long as you use natbib style citation commands.
Or you can replace all your citet and citep commands
with textcite and parencite commands.
Traditional spreadsheets:
It has become very popular to use spreadsheets in education.
This has some downsides.
Traditional preadsheets are OK for very simple, one-off applications using small amounts of data.
For serious work,
they are too opaque, because they are too hard to document.
(If you have ever tried to figure out another person's complex spreadsheet, you know what I mean.)
Particulary well publicized was the Excel error whose correction
overturned a much cited Reinhart and Rogoff study.
But studies indicate that almost all large spreadsheet have serious errors.1,2
Basic data handling glitches have been documented even in the most popular spreadsheet applications.
The lack of transparent scripting facilities means that it is very difficult to understand someone else's spreadsheet or even one’s own spreadsheet developed months or years ago.
If all you want to educate someone to do is one-time, never-to-be-revisited, quick-and-dirty computations,
then teach them a traditional spreadsheet.
Otherwise teach them a format that allows serious documentation and transparency of manipulations.
At the very least,
eschew traditional spreadsheets in favor of newer alternatives,
such as scientific spreadsheets and engineering spreadsheets.
Comment: some problems with spreadsheets can in principle be overcome by using them directly just for data storage and then preforming manipulations via a scripting language that exploits the spreadsheet's computational and graphics facilities---on a Windows platform, perhaps VBA or better yet Python with win32com; on a Unix platform, perhaps ABV or Python or a plug-in that supports scripting. Of course this is not how most people use spreadsheets. There are also engineering spreadsheets that allow explicit scripting and treat data in a more object oriented fashion. These are tools that can be properly used for serious work.
Comment: Python can read and write tranditional spreadsheets. For simple spreadsheets see the CSV module. For more advanced needs, consider xlrd and xlwt.
If you are interested in spreadsheet structures, you may be interested in Fenfire generally and ZigZag more specifically.
Perhaps the simple yet powerful NoSQL will meet your needs. More traditional approaches follow.
SQLite "is a small C library that implements a self-contained, embeddable, zero-configuration SQL database engine". (License: Public Domain.)
FireBird "is a relational database offering many ANSI SQL standard features that runs on Linux, Windows, and a variety of Unix platforms."
OpenOffice includes a database. GNOME Office includes GNOME-DB, which is reputed to be very good. For enterprise needs, PostgreSQL has a good reputation: PostgreSQL is a free-and-open-source, SQL-compliant, object-relational database management system, reputed to be highly-scalable. Using Ruby with Rails and say MariaDB or PostgreSQL is getting a lot of buzz for developing data-base backed web applications.
Simple but structured needs may even be met by Open-RJ.
faces is a user friendly project management tool. (License: GPL; Language: Python)
Task Coach is a lightweight task manager. (License: GPL; Language: Python)
Task Juggler is very popular. (License: GPL)
OpenOffice includes a database. GNOME Office includes MrProject, "a project planning, scheduling and tracking tool.
Consider HDF5 for substantial databases. (Overview.) Binaries. Windows installation.
Scribus is a featureful, cross-platform desktop publishing application. Supports PDF 1.4 and Python scripting. "The Scribus file format is XML based; open and completely documented." (License: GPL) Used by professionals to send multipage full color layouts sent directly to commercial print houses. Supports CMYK separation, PDF generation, and much of the toolsets you get from commercial packages like Quark or InDesign.
HylaFAX offers an open source version for a variety of *nix platforms. (This OpenOffice setup discussion may be helpful.) I do not know of any FOSS fax software for Windows or Mac: let me know if you do!
Pidgin IM (formerly Gaim) is a free and open source instant messaging client for a variety of platforms. ScatterChat is a secure enhancement of gaim.
Mac users can consider Fire, which supports most popular instant messaging services. (GPL)
A compose key allows inputting special characters as key combinations. Some text editors provide support for this (e.g., digraphs in Vim). Linux includes compose-key activation as a native ability. For Windows users, a few are provided via the Alt key and the numeric keypad. However, WinCompose provides better functionality. (License: WTFPL (essentially public domain).) (There is a Youtube tutorial for WinCompose.)
Calculators are still useful. Here are a few of many.
Reverse Polish Notation:
The dc desktop calculator,
for those comfortable in a shell (including cmd.exe),
is cross-platform, wonderfully fast, and ever handy---highly recommended!
Another cross platform possibility is kalc.
For Linux you can try rpncalc.
For Windows I like the freeware
RPNCalc,
which is also ported to WinCE.
RPN is an interesting calculator for the PalmOS.
And finally for PostScript enthusiasts,
I cannot refrain from mentioning PSCalc.
For a console calculator, try the cross-platform bc. (The bc manual is very good.) For a graphics calculator, try the cross-platform GraphCalc. Or just use gnuplot as a graphics calculator. GNU Emacs Calc has many features, including graphics using GNUPLOT. CoCalc is a freeware JAVA calculator.
Rascal is enough more than a calculator that it may be considered a simple matrix programming language. This is also true of cxc, and Iguana seems to be evolving in the same direction.
Check TheFreeCountry for a longer list.
Python users get a basic cross-platform solution automatically via the ftplib module. (For an even lighter approach, try this TFTP module.) The Python Cookbook uses this module to provide a threaded ftp client. As of August 2005, ftplib does not provide any secure services, however. For SSH2 services you can use paramiko, which relies on pycrypto. These are very easy installs! (Get paramiko from the releases tab on GitHub, and see the README file for instructions. Windows users can also look at Voidspace for prebuilt PyCrypto binaries.) I provide an example (usable) SFTP client using these modules, but you can even have a graphical interface with SFTP support via wxSFTP. (If your needs are more substantial, you can use pycurl as a Python interface to libcurl, which should be able to meet all your ftp needs.)
If you need FTP/S, you can try TSLite, which offers a pure Python solution for FTP over TSL.
For Windows: I find the free and open source FileZilla to be excellent for FTP and SFTP.
(In older versions, I have had some problems with FTP/S in passive mode.)
Other possibilities:
WinSCP.
OpenSSH for Windows.
WS FTP
had a free (for academics and students), easy to use, ftp program for Windows platforms,
but I think the free version went away.
For Mac:
CyberDuck.
For Un*x:
NcFTP.
(Also runs on Windows and Mac, for those willing to learn ftp commands.)
OpenSSH.
GOCR, Kooka, OCRShop, ClaraOCR
Generating reports is a common need. These can be informal or presentational, and may or may not need to be printed.
gawk, a powerful report generation language which takes only a moment to learn well enough to produce simple reports. (E.g., you can manage student grades with Gawk much more easily than with a spreadsheet.)
ReportLab allows direct creation of PDF graphics and reports.
iText can be a very powerful report generator if you know a bit of Java.
Apache Formatting Objects Process (FOP) offers amazing flexibility. Java based. (License: Apache 2.0)
Program controlled printing can be a challenge, especially under Windows. If you have the option of producing your reports as PDF or as appropriately formatted text, then you can usually just ask your operating system to use a default application to print them. Otherwise, you will have to work harder.
Tim Golden offers a very useful discussion of Windows printing under Python.
GNU a2ps allows conversion of text (and other) files to PostScript, as does enscript and ASCIIprint.
wordaxe can be used for hyphenation. For example, you may want to generate your own PostScript or PDF. As a more specific example, the wordaxe hyphenation library offers ReportLab support. For an easy interface, try rst2pdf, which offers cascading stylesheets and supports TTF and Type1 font embedding.
There are many version control systems (VCSs): see the Wikipedia list. If you just need revision control for a single-user, single-machine project, you can use RCS. For simple version control on multi-user or multi-machine projects, consider Subversion (SVN). Currently distributed VCSs are the rage, with Bazaar (License: GPL), Mercurial (License: GPL), and especially Git (License: GPL) having particularly high profiles. (Windows users of Git should consider GitHub for Windows.) Having a distributed VCS is not important for most small projects.
See my Subversion hints.
Despite the recent popularity of Git,
Subversion is probably the most widely used version control system.
There is an extensive user manual.
(Be sure to read about the basic work cycle.
Simplest bottom line: change to your working directory and enter
svn update, then make changes to your files.
Before you commit do another update,
and finally commit the changes with svn commit -m "My message about the changes.".)
The pysvn project provides a Python based graphical interface to SVN.
SVN on windows: download and execute the Windows installer. Users who dislike the command line should consider TortoiseSVN as a user-friendly Subversion client.
SVN with Vim: vcscommand provide keybindings for SVN control from the Vim editor.
Free and low cost SVN repositories:
assembla
(includes some free options)
Unfuddle
Subversion users may find it helpful to generate graphical reports of the repository changes. A Python-based solution is svnplot. A Java-based solution is StatsSVN.
Track stolen laptops and cell phones with Prey.
Calibre has quite amazing conversion capabilities and supports many e-book formats.
Outlines can help writers and programmers.[ 1, 2 ] Many good editors offer substantial outlining capability. Examples include the VimOutliner and the emacs outline mode.
There are many commercial and free outliners. The Java Outline Editor offers serious outline functionality.
FreePlane is an active fork of FreeMind, which is a highly capable cross-platform mind mapper. It is Java and includes and Windows installer. (License: GPL) For a more document oriented approach, try Kdissert. (License: GPL) MindMup provides free online mind mapping, including online collaboration. (License: MIT. Supported by Google Drive.)
Tree Sheets is an odd cross between a spreadsheet and a mind mapper.
The basic situation. The graphics problem.
The hoary standard is m4. An interesting alternative is empy. Good luck.
After the Windows and Mac operating systems, most people will think first of GNU Linux, an excellent, free, Open Source version of the Unix operating system. Another popular free and open source Unix relative is FreeBSD. Another popular version of the Unix operating system is Solaris, a FOSS version of which is available as illumos (formerly OpenSolaris).
Plan 9 is very roughly a faster, more secure multi-processor Unix. There is an open source implementation of BeOS called Haiku, but in 2015 it seems that organization issues may lead to a forked project calles Senryu. (Haiku can install and run within Windows.) QNX's Neutrino Real Time Operating System is also a compact and stable operating systems. (Can install and run within Windows, and thus does not require a separate partition or boot loader!) An interesting effort to produce an open source (and binary compatible) substitute for Windows NT is ReactOS, which already has basic console functionality. A few people may also be interested in FreeDOS.
Things have gotten better with the advent of PowerShell, but Windows users still suffer from a dearth of good shells. Here are a few options.
Fink has an extensive collection of Unix tools for Mac users.
Note that password protecting your Word files does not encrypt them. Password protection of PDF files is easily cracked, although Adobe requires that readers respect the protection. Some editors (e.g., Vim) allow easy, strong encryption of your text files.
Banshee music player. (License: MIT)
CDex CD ripper. (License: GPL) On Windows, CDex may provide an alternative to the poor iTunes error correction (i.e., the presence of pops and clicks).
Tomboy note-taking application.
Please let us stop playing nationalist games with number formats before someone gets badly hurt as a result. For scientific data exchange, the decimal separator is a point (ASCII 46). It is not a comma (ASCII 44). Neither is it any other UTF code point. It is just that simple. Pretending otherwise is like pretending that science does not use the metric system.
Possibly the simplest open standard for fixed periodicity time-series data is the open-databank standard. Most time series econometrics packages support some version of this.
The most widely used data exchange format is comma-separated values (CSV). There actually is a standard, in the form of RFC 4180. Sadly, many files with the .csv extension do not conform to this standard, so caveat emptor.
AstroAsciiData provides simple database functionality for ASCII tables.
HDF5 "is a general purpose library and file format for storing scientific data". Some support can be found in the econometrics community, both in commercial and free packages. Python users have excellent support via PyTables. (To use PyTables with Python 2.4 in Windows platforms, you will need to get the Windows binaries.) Octave and R provide HDF5 support. Commercial applications supporting HDF5 include Mathematica and MATLAB version 6.5+, among many others.
The netCDF libraries "support a machine-independent format for representing scientific data". The netCDF distribution includes interfaces for C, Fortran77, Fortran90, and C++, and interfaces are reportedly available for Ada, IDL, Java, MATLAB, perl, Python, R, Ruby, Tcl/Tk. The interface provided by ScientificPython is supposed to be excellent. PyNIO adds to this support of additional formats. Another possibly faster way to read the netCDF format is pupynere, a pure Python solution. (License: MIT)
CSV stands for comma-separated values. Distinguish CSV and DSV. CSV is described by RFC 4180. Do not give a .csv extension to files that fail to match this description.
Be sure to set your client to produce plain text email. I have written up a few reason and clues.
Make Vim your editor and call it as
c:\programs\vim\vim73\gvim.exe -c "so $VIM\vimfiles\mail.vim" %s
where mail.vim is where you put any mail processing commands
(e.g., set ft=mail tw=75).
You are welcome to use my mail commands.
Under OS X Mahogany uses the Unix OpenSSL library, so you need to: 1. install openssl libraries (you can get them from fink) 2. point M to then: in the "helpers" page of the options dialog, add the correct paths to them, then it should work
\pathto\gvim --nofork -c "setf mail",
so you will get 'mail' syntax highlighting and ftplugins loaded.
Or if you have defined say mail.vim,
a file of useful macro,
you can instead use e.g.,
\pathto\gvim.exe --nofork -c "so $VIM\vimfiles\mail.vim".
After installing the extension,
edit the composition toolbar (the one for replying to email, not the main application toolbar)
and add a button for the external editor.
You can then click that button and your current email reply will be opened in Vim.
%VIM%\vimfiles folder,
and change your External Editor command to, e.g.,
"C:\Program Files (x86)\Vim\vim82\gvim.exe" --nofork -c "so $VIM\vimfiles\ai_mail.vim".
d key with goDoCommand('cmd_delete');
and the x key with goDoCommand('cmd_MsgCompactFolder');.
If you prefer to POP your mail, you can get more control over this than Pine offers with radiks, a tiny mail transfer program (MTA) for win32 (Windows 9x/NT/2000) which reads mail from a pop3 server and transfers it (filtered, if you want) to your local hard drive.
Use the Lapack routines if you need speed. E.g., for a QR decomposition use scipy.linalg.flapack.dgeqrf instead of scipy.linalg.qr.
def dot(a, b):
return numarray.dot(a, b.view())
The original b is then not touched, and the screwed-up matrix is
discarded.
import sys
sys.path.remove('')
import sets
to remove the current directory from Python's search path.
If you then need access to any modules in the current directory,
use sys.path.insert(0,'')" first.
As late as 2004,
the Numeric and numarray documentation incorrectly stated that
and, or, and not would
operate element-by-element on arrays.
Lists are sorted in place, so mylist.sort() return None,
but mylist is now sorted.
Rossiter wrote an accesible Introduction to the R Project.
Documentation:
Windows users may want to read R Guide for Windows Users by Kevin Wang.
Lund University offers some supplementary documentation;
also Harrell's R utilities along with his guide to LaTeX tables and S may be useful.
Ikaha offers some introductory discussion of generating graphics in R.
Be careful with the definition of R’s PRNG for the geometric distribution:
rgeom has support on nonnegative integers,
since R has p(x) = p (1-p)^x.
A perhaps more common parameterization is p(x) = p (1-p)^(x-1):
which you can match by adding 1 to the values returned by rgeom.
You can write LaTeX documents containing R code whose ouput is to be included in the document with Sweave.
>>> from win32com.client import Dispatch
>>> sc=Dispatch("StatConnectorSrv.StatConnector")
>>> sc.Init("R")
>>> print(sc.Evaluate("2+3"))
5.0
>>>
Occasionally you will need to provide or adjust the bounding box information for a graphic. Here are some resources.
gs -sDEVICE=bbox -dNOPAUSE -dBATCH file.ps
will print the bounding box information for file.ps (and then exit gs).
You can then use this information to reset the bounding box.
bbox_add.pl is a perl script that uses this approach.
fixBBox.py is a Python script that uses this approach.
(I have not tried these scripts.)
abiword --to=latex test.abw
pdflatex test.latex
<Scheme
name="AbiVi"
KeyBindings="viEdit"
KeyBindingsCycle="1"
/>
<Select
scheme="AbiVi"
autosaveprefs="1"
useenvlocale="1"
/>
When you add comments to a document, the comments are stamped with a name, a date, and a time. This may well be more information than you want to provide. You can remove the name stamp from all your comments as follows. Show the comment list: pick Comments » Show Comment List from the menus. Select all comments: click on one, and then press ``ctrl-a``. Change the comment properties: right click on your selection, click ``Properties``, pick the ``General`` tab, and change the Author field (e.g., nothing). Unfortunately, you cannot remove the date and time stamps this way. However, only the "modified" stamp shows to the reader, when you change the author this way, it should update the modified time on all the comments.
print /flush;;
to force GAUSS to flush the buffer.
You can also use the "configure/preferences/cmd_window" menu
and uncheck the box for "Buffer output".
Please note that files can only be downloaded via COMMAND LINE FTP, not
a browser (such as Internet Explorer, Netscape, Firefox, etc.). You
must use a command window to download the files. To open a command
prompt window, click on Start, then Run, and type cmd.
GAUSS Light 8.0 for Windows
---------------------------
Below is a list of files that you may wish to download and
what each is:
GAUSS_Light_8.0_Win.zip [GAUSS Light 8.0 software]
GAUSS_8.0_Manual.zip [all documentation zipped together]
Save the files that you download to either your root directory
(usually C:) or a directory that you set up that has no spaces
in the title; there can be problems saving the files to a
directory such as C:\Documents and Settings\<username>.
You *must* know the name of the files you wish to download.
DO NOT USE A BROWSER TO DOWNLOAD THE FILES. To download
the necessary files via command line ftp, do the following
at a terminal prompt:
1. Type the following at the ftp prompt:
ftp ftp.aptech.com
2. Once you are in the ftp.aptech.com site, type:
anonymous
then when prompted enter your email address as the password.
3. Type the following at the ftp prompt to get the files
(ftp> represents the ftp prompt you should see, do not type that):
ftp> get
(remote-file) /outgoing/GAUSS_Light_8.0_Win.zip
(local-file) GAUSS_Light_8.0_Win.zip
ftp> get
(remote-file) /outgoing/GAUSS_8.0_Manual.zip
(local-file) GAUSS_8.0_Manual.zip
ftp> bye
Approximate file sizes
----------------------
8.8 MB GAUSS_Light_8.0_Win_32.zip
8.5 MB GAUSS_8.0_Manual.zip
========================================================================
========================================================================
Please note that files can only be downloaded via COMMAND LINE FTP, not
a browser (such as Internet Explorer, Netscape, Firefox, etc.). You
must use a command window to download the files. To open a command
prompt window, click on Start, then Run, and type cmd.
GAUSS Light 8.0 for Mac Universal
---------------------------------
Below is a list of files that you may wish to download and
what each is:
GAUSS_Light_8.0_Mac_OS_Xu.dmg [GAUSS Light 8.0 software]
GAUSS_8.0_Manual.zip [all documentation zipped together]
Save the files that you download to either your root directory
(usually C:) or a directory that you set up that has no spaces
in the title; there can be problems saving the files to a
directory such as C:\Documents and Settings\<username>.
You *must* know the name of the files you wish to download.
DO NOT USE A BROWSER TO DOWNLOAD THE FILES. To download
the necessary files via command line ftp, do the following
at a terminal prompt:
1. Type the following at the ftp prompt:
ftp ftp.aptech.com
2. Once you are in the ftp.aptech.com site, type:
anonymous
then when prompted enter your email address as the password.
3. Type the following at the ftp prompt to get the files
(ftp> represents the ftp prompt you should see, do not type that):
ftp> get
(remote-file) /outgoing/GAUSS_Light_8.0_Mac_OS_Xu.dmg
(local-file) GAUSS_Light_8.0_Mac_OS_Xu.dmg
ftp> get
(remote-file) /outgoing/GAUSS_8.0_Manual.zip
(local-file) GAUSS_8.0_Manual.zip
ftp> bye
Approximate file sizes
----------------------
13.9 MB GAUSS_Light_8.0_Mac_OS_Xu.dmg
8.5 MB GAUSS_8.0_Manual.zip
proc quantilenew(x,e,a,b,c,d);
local w, wt, f, r, i, z;
if minc(e) < 0 or maxc(e) > 1;
errorlog "Inadmissable quantile levels";
if not trapchk(1);
end;
else;
retp(error(0));
endif;
endif;
w = a + (rows(x) + b) * e;
wt = floor(w);
f = c + d*(w - wt);
if wt == 0 or wt == rows(x);
errorlog ftos((maxc(1/minc(e)|1/(1-maxc(e)))),"*.*lf",1,0)
$+ " rows required for this level";
if not trapchk(1);
end;
else;
retp(error(0));
endif;
endif;
r = zeros(rows(e),cols(x));
i = 1;
do until i > cols(x);
z = sortc(x[.,i],1);
r[.,i] = z[wt] + f .* (z[wt+1] - z[wt]);
i = i + 1;
endo;
retp(r);
endp;
** =========================================================================== ** Procedure Format Purpose Line ** =========================================================================== ** inthp1 y = inthp1(&f,pds,ctl); integration over -inf,+inf ** inthp2 y = inthp2(&f,pds,ctl,a); integration over a, +inf non-oscil ** inthp3 y = inthp3(&f,pds,ctl,a); integration over a, +inf oscill ** inthp4 y = inthp3(&f,pds,ctl,c); integration over a, b ** ===========================================================================
atranspose,
so that the two dimensions that you want in your matrix are in the two last dimensions of the array.
Then extract the matrix with getmatrix using the appropriate locator.
trap and scalerr commands.
(See the example for 'scalerr' in the GAUSS Command Reference.)
complex(b,0)^x,
where b<0 is the base and x is the exponent.
let string a[4,4]="";.
But what if the dimensions r, c are not known ahead of time?
Then use a=""$+zeros(r,c); instead.
call setvwrmode("many");
call setvwrmode("one");
_pcolor.
If you do not like the default colors
for the graph characteristics (ticks, frame, labels, background, etc)
you can change them:
see the help for _pmcolor.
_pcmdlin.
To convert a graph into a ps file (flag -c ) stored in
"c:\filename.eps" (flag -cf ), include the line
_pcmdlin = "-c=1 -cf=c:\\filename.eps";
cvt_color = RGB if you want color graphs.)
_ptek global,
and then use the tkf2eps function to convert the .tkf files to Encapsulated Postscript.
The syntax is: ret = tkf2ps("mytekfile.tkf", "myepsfile.eps");
(see the GAUSS Command Reference).
i = sqrt(-1);x=2-3*i;.
Note that GAUSS does not set the value of i by default;
and you can pick any variable you want for this purpose.
for command in GAUSS the loop indicator is not a floating point.
Up to v.3.2.44, you need a hack if you want to pass the loop indicator to a procedure
(e.g., to convert it to a character value). For example, the following code works.
for i (1,10,1);
j= i;
print $ftocv(i+0,8,0) $ftocv(j,8,0);
endfor;
for loops is not safe in recursive procedures.
This is supposed to be fixed in version 3.7 or later.
if abs(imag(x)) ≤ 1e-12 ;
x = real(x);
endif;
For real symmetric matrices,
be sure to compute eigenvalues with eigh instead of eig:
this guarantees real eigenvalues and is much faster than eig.
Furthermore, if you use eig() for a real symmetric matrix,
rounding error in the calculations will nearly always produce complex
eigenvalues with imaginary parts that are very small.
cdfchii through GAUSS 4.0 and possibly later.
Test with the following code snippet:
cdfchii(0.9998,13);
As one might expect, the same problem arises for the inverse gamma function.
Test with the code snippet 2*gammaii(13/2,0.9998);
let x1 = 4;
let x2 = a;
s1 = "" $+ x1;
s2 = "" $+ x2;
if (s1 $== x1); "ok"; endif;
if (s2 $== x2); "ok"; endif;
Only the second equivalence (s2 $== x2) is true.
y = y * x + diagrv(zeros(rc,rc),c[i]);
y = y * x + diagrv(zeros(rx,rx),c[i]); @changed rc to rx@
#define A print "---"; #ifdef A /*space ^ */ /*here | */ print "A"; #endif print "===";yields
--- ===as its output.
varput() and ftocv().
For example,
for i(1,5,1);
varput(eye(i),"x"$+ftocv(i,1,0));
endfor;
c:\gauss\tgauss.exe -b c:\mypath\myfile.gau
where you need to adjust the paths appropriately.
The -b flag is used to call GAUSS in batch mode.
Save the batch file (e.g., as filename.bat).
As with any batch file, you can double click it to run it
or run it from the command line.
sa = ftostrC(x,"%8.4lf");
fp = fopen("matrix.asc","w");
call fputst(fp,sa);
fp = close(fp);
c:\programs\aspell\bin\aspell check c:\temp.tex
(assuming that's where you put it, of course).
You may want to change your path so that aspell is in it.
map <f7> <esc>:w<cr>:!aspell check %<cr>:e! %<cr>
map <Leader>s <esc>:w<cr>:!aspell --dont-backup -c %<cr>:e! %<cr><cr> c:\Python27\python.exe -c "import scipy;scipy.test()" >c:\file.txt 2>&1In PowerShell, use the
out-file commandlet.
svn co https://pyx.svn.sourceforge.net/svnroot/pyx/trunk/pyx pyx
and then change to your new pyx directory and enter setup.py install.
mygraph.writeEPSfile("c:/mydir/myfig.eps",paperformat=pyx.document.paperformat.Letter)
bboxenlarge=False to ensure a "tight" bounding box.
For 2-D data plots, Titus Winters has written a nice PyX tutorial.
Be sure to see the examples and gallery on the PyX website. I provide a heavily commented elaboration of the shaded integral example.
For a simplified interface to PyX, you can add PyXgraph or, if you would like an interactive command line interface, PyXPlot. For geometry, geoPyX is limited but useful.)
D "mgs.exe -q -sPAPERSIZE=a0 -sDEVICE=pdfwrite -dCompatibilityLevel=1.2 -dUseFlateCompression=true -dNOPAUSE -sOutputFile=\"%o\" \"%i\" -c quit"Some applications (e.g., PyX) produce negative bounding boxes. (See Pyx hints for a PyX solution.) While perfectly legal, this creates a problem for dvipdfm. During conversion to PDF, the EPS files are clipped at the papersize boundary rather than the figure boundary. I don't know how to force pdfwrite to behave nicely, and epstopdf does not work because it intentionally alters the bounding box for the PDF file. You can try it though by changing the MiKTeX configuration to:
D "epstopdf --outfile=\"%o\" \"%i\" "
\begin{figure}[ht]
\special{pdf:image (mydir/myfile.pdf)}
\end{figure}
$File in the Parameters box with
&latex $FileName.
(MikTeX users can simply use $FileName.)
Click on "Apply"
To LaTeX (the disk copy of!) your current document, select your TeX from the "Tools" pull-down menu.
IMPORTANT: When you LaTeX a document in TextPad, a Command Results document is opened.
When LaTeX finishes executing, it displays the message:
Process completed with exit code 1.
YOU MUST CLOSE THIS WINDOW AFTER EVERY RUN.
Failure to do this will cause subsequent LaTeX or BibTeX runs to hang.
To avoid the afore mentioned problem, I recommend running MikTeX as a DOS command, which can also be done as a TextPad tool.
If you decide on this option, go to the TextPad menus Configure, Preferences, Tools.
Click on add and select DOS Command.
A window will appear in which you type the full path to the
command you wish to execute. For example, for MiKTeX type:
c:\texmf\miktex\bin\texify.exe $BaseName
(If you use the default MiKTeX installation, you do not need a fully qualified path name.)
Click OK, then Apply,
then go to the tool you just added
(it should be listed under Tools when you expand it)
and make sure to UNcheck the "capture output" box.
If you do all of this, then when you select the tool, a DOS window will pop open and show you the execution results. This is useful for when you have compilation errors,
you can hit enter, "X" or whatever,
whereas the other way you can't.
Older versions of TextPad have a small bug:
the DOS window closes upon the end of execution of your tool, whether the close window box is clicked or not.
Note that the texify command offered in MikTeX automatically runs BibTex for you.
$File in the Parameters box with
$BaseName.
Click on "Apply"
To LaTeX your current document, select Bibtex32 from the "Tools" pull-down menu.
IMPORTANT: When you LaTeX a document in TextPad, a Command Results document is opened.
When LaTeX finishes executing, it displays the message:
Process completed with exit code 1.
YOU MUST CLOSE THIS WINDOW AFTER EVERY RUN.
Failure to do this will cause subsequent LaTeX or BibTeX runs to hang.
$File in the Parameters box with
$BaseName.
(TrueTeX users will need to use $UNIXDir/$BaseName.)
Click on "Apply"
To view your LaTeXed document, select your viewer from the Tools pull-down menu.
$File in the Parameters box with
$BaseName.pdf.
Click on "Apply"
To view your LaTeXed document, select your viewer from the Tools pull-down menu.
Reassign Ctrl-e to WordRightEnd as follows: choose Customize from the Configure menu, select the Keyboard tab, choose Cursor from the Categories list, choose WordRightEnd from the Commands list, type Ctrl+e in the Press New Shortcut box, click Assign, and click OK. You will then want to assign Ctrl+Shift+E to WordRightEndExtend.
Get all secion header cells with Cells[CellStyle → {"Section"}].
The result is a list of cell objects,
on which you can set any options by assigning to its CurrentValue:
Do[
CurrentValue[cell, ShowGroupOpener] = True,
{cell, Cells[CellStyle → {"Section"}]}
]
Format » Option Inspector... in the menu.
In the top-left dropdown, set the option scope Selected Notebook.
Find FileOutlineCache under Notebook Options » Private Notebook Options,
and set this option to False.
(This may make opening the notebooks a bit slower.)
Similarly,
under Editing Options,
set TrackCellChangeTimes to False
SetOptions[InputNotebook[],
PrivateNotebookOptions → {"FileOutlineCache" → False},
TrackCellChangeTimes → False
]
Save your notebook and it will save with much less clutter
and many fewer changes.
To do even better,
as a preliminary to committing your changes,
delete all your output cells and then save your notebook.
(You can use the menus or FrontEndTokenExecute["DeleteGeneratedCells"] for this.)
On campus: To fix font problems on the display, make sure the Windows Multilingual Support is installed. If that is not an option, go to Tools, User Setup, Font Mapping and check the Use Symbol Font mapping box.
Be sure to learn the keyboard short-cuts: see especially the help for Key Prefixes. For example, you can logical negation as ctrl-s,N and the partial derivative sign (∂) as ctrl-s,d. (See the Help search.) Also, I strongly recommend changing the default settings under Tools, UserSetup, Math: set it so that two spaces switches to math and a space at the end of math switches to text.
Be sure to visit Yngve Svendsen's Scientific WorkPlace Unofficial Homepage (although I have had some connection difficulties).
Graphics: When using the JPGs or TIFFs, edit the file tcispech.ini and change the setting for MaxGraphicsMemoryUsage. Try doubling or tripling the default setting of 4000. When using EPS graphics, save your document using the Portable LaTeX filter and change the options for the graphicx package to use dvips. The graphics will only appear when printing, but they should appear.
To use Scientific Word on the EagleNet, you will first need to install some (non-executable) files on your computer (which ought to have say 30Meg free disk space and 8Meg RAM); do this by entering "enetupgr sword" (without the quotes) at the DOS prompt. (Note: be sure to keep your macros in a separate file, which you \input in your preamble.) Be sure to see the Scientific Workplace Unofficial Homepage, which include instructions on how to use MiKTeX with SWP.
You can get all the functionality of Scientific Word
except for the typesetting and associated TeX functions in
Scientific Notebook for Windows 9x and NT.
Many useful functions are in the
free Demo version, although this version cannot print, save, or copy from read/write documents.
While you are on the AU campus, you are in luck:
Scientific Notebook is available as an EagleNet Application.
Users must first open EagleNet Applications.
(If you don't have EagleNet on your Start menu,
just pick Start, Run and the type `nal' in the dialog box.)
You should see a Math/Statistics/Research
folder on the left, in which you will find the
Scientific Notebook icon.
Double click the icon to run the program.
The first time you run the program, several system files
will be copied down to your hard drive before the program opens.
Each time after that initial setup the program will run
as soon as the icon is double-clicked.
The program is accessible to the major academic user groups
and all student users.
Since it is a 5-user license, you may occasionally find you have to wait until someone logs off to use Scientific Notebook. I do not expect this to happen, but if it happens to you let me know and I will apply for more copies.
Subversion is easy to use and excellent for individuals and small groups. Here is a ten minute tutorial. Windows users who prefer to avoid the command line can just use TortoiseSVN.
How to resolve Subversion Conflicts
svn add --non-recursive mydir.
One disappointing thing: it would be nice to add a directory
and all its files except those matching the svn:ignore property.
Subversion does not currently support this.
Here is one work around.
You can set an editor (e.g., for commit messages and property editing)
in the Subversion config file.
On Windows you should be able (after installing Subversion) to find it here:
\Users\YourUserName\AppData\Roaming\Subversion.
Search for editor-cmd and follow the instructions.
(On Windows, I am currently using editor-cmd = C:/programs/Vim/vim74/gvim.exe -f.
If you use a path with spaces in it, you will have to quote it.)
Once you ensure that Subversion has access to a text editor,
you can edit the list of patterns it will ignore in any directory.
E.g., svn propedit svn:ignore . (note the final period)
will edit the patterns for the current directory.
svn diff -r HEAD myfile shows differences between myfile and the repository.
svn status --show-updates shows you which files in the repository have changed.
It is useful (and probably best practice) for any plain text files to
svn propset svn:eol-style=native *filename*.
This is not the default, but each user can make it a personal default by editing
the [autoprops] of their Subversion config file.
From Subversion 1.8 onwards, it can be set as a default at the repository level
with Repository Dictated Configuration.
It is a good idea to lock binary files, to help ensure only one person changes them at a time:
svn propset:needs-lock yes *filename*.
\usepackage[T1]{fontenc} .
Unfortunately, on its own, this will probably lead to the
use of bitmap fonts and thus "fuzzy" screen output.
According to Guenter Milde,
there are several reasons for this choice.
\usepackage{lmodern})
or by using a Docutils command line argument
(--stylesheet=lmodern).
Another popular solution is the "Times/Helvetica/Courier" combination,
which you can get by adding the following to your style sheet:
\usepackage{mathptmx} % Times for serif and math
\usepackage[scaled=.90]{helvet} % downscaled Helvetica for sans serif
\usepackage{courier} % Courier for teletype (mono-space)
or with the following Docutils command line options:
--documentoptions="scaled=.90" --stylesheet="mathptmx,helvet,courier".
--font-encoding=OT1
(or even --font-encoding="").
and
(*not* by commas).
Required tab
and the issue number is entered on the Optional tab.
General tab.
Matplotlib is included in many Python distributions, including Enthought Canopy. If you install it your self, be sure to read the installation instructions. Before installing Matplotlib, you will certainly need to install NumPy.
To get the most from Matplotlib,
you need to clearly understand the difference
between a figure instance and an axes instance.
A figure instance the basic container for an entire chart,
like a figure in a journal article.
A figure instance can contain multiple axes instances,
each of which can be used for plotting or drawing.
We can call the add_subplot method to add
non-overlapping axes instances to our figure.
(But if you call add_subplot again
in a way that overlaps your previous subplot,
this **replaces** your previous axes in the current figure.)
import matplotlib.pyplot as plt fig1 = plt.figure() ax1 = fig1.add_subplot(1,1,1) #now draw on ax1 fig2 = plt.figure() ax2a = fig2.add_subplot(2,1,1) #now draw on ax2a ax2b = fig2.add_subplot(2,1,2) #now draw on ax2b
You can use the subplots convenience function
to do figure and axes creation at one go.
fig, ((ax1,ax2),(ax3,ax4)) = plt.subplots(2,2)
Here is a simple example for creating a figure and displaying it onscreen. (For completeness, we illustrate the explicit choce of a GUI “back end”, although this is usually not necessary.) Note the hierarchical approach: the pyplot interface creates the figure object, the figure object creates the axes objects, and the axes objects create the other objects (including, implicitly, the axis objects).
# preliminaries
import matplotlib
matplotlib.use('TkAgg') #note choice of GUI backend!
import matplotlib.pyplot as plt
# create a new figure
fig = plt.figure(1)
ax = fig.add_subplot(1,1,1)
ax.plot([1,2,3],'r-')
ax.set_title('Easy Line Plot')
# display your figure onscreen
plt.show()
Here is a simple example for creating a figure and saving it, without displaying it onscreen. (I.e., no GUI.)
# preliminaries
import numpy
import matplotlib
matplotlib.use('Agg') #note choice of backend: no GUI!
import matplotlib.pyplot as plt
#create some artificial data for this example
x, y = numpy.random.rand(2,50) #the data (50 points)
# create a new figure
fig = pyplot.figure()
#add an axes instance to draw on
ax = fig.add_subplot(111)
#do your drawing
ax.plot(x, y, marker='.', linestyle='', markerfacecolor='blue')
ax.set_title('Scatter dots')
ax.set_xlabel('x')
ax.set_ylabel('y')
# save figure to file
fig.savefig('/temp/myfig.png', dpi=96)
Matplotlib's pyplot facilities are convenience facilities for the user with modest needs. For some examples (e.g., embedding Matplotlib in other applications) it is important not to use these facilities. In this case, you can altogether avoid using pyplot, but you will need to handle figure and canvas creation yourself.
# example by John Hunter.
from matplotlib.backends.backend_svg import FigureCanvasSVG
from matplotlib.figure import Figure
fig = Figure()
ax = fig.add_subplot(211)
ax.plot([1,2,3])
ax.set_xlabel('time')
canvas = FigureCanvasSVG(fig)
canvas.print_figure('myfile.svg')
As another example, here is our earlier example redone to be more fully object oriented.
# preliminaries
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from matplotlib.figure import Figure
# create a new figure
fig = Figure()
ax = fig.add_subplot(111)
ax.plot([1,2,3],'r-')
ax.set_title('Easy OO Line Plot')
ax.set_xlabel('x')
ax.set_ylabel('y')
canvas = FigureCanvas(fig)
canvas.print_figure('temp.eps')
All Matplotlib plotting functions trigger autoscaling.
This affect, e.g., setting the axes limits.
To prevent your limits from being overridden,
call xlim/ylim after any plotting functions.
Alternatively, you can turn autoscaling off:
gca().set_autoscale_on(False)
Figure.subplots_adjust,
like this:
figure() subplots_adjust(hspace=0.4) # default: 0.2 subplot(211) subplot(212)
Fans of Edward Tufte may wish to look at etframes.
Note that if you set_aspect on a axes instance you
will still need to apply_aspect as well
(or call draw).
Matplotlib is capable of fully object oriented graphics programming. There is a very useful tutorial. If you want to embed Matplotlib in a GUI, you must use the object oriented API and not pyplot. The Matplotlib website includes examples of embedding. (For example, embedding_in_qt.py.)
wxmpl is supposed to be the best way to integrate Matplotlib into wx.
import matplotlib
matplotlib.use('GTKAgg')
from matplotlib.pyplot import *
# GTK backend
from pylab import *
import gtk
figure(1)
plot([1,2,3])
manager = get_current_fig_manager()
# see gtk.Window class docs at
# http://www.pygtk.org/pygtk2reference/class-gtkwindow.html
manager.window.set_position(gtk.WIN_POS_CENTER)
figure(2)
plot([1,2,3])
manager = get_current_fig_manager()
# see gtk.Window class docs at
# http://www.pygtk.org/pygtk2reference/class-gtkwindow.html
manager.window.set_position(gtk.WIN_POS_NONE)
show()
Another user offered a WX example:
# WX* backend (manager.window is a wxFrame)
# http://www.lpthe.jussieu.fr/~zeitlin/wxWindows/docs/wxwin_wxframe.html#wxframe
figure(1)
plot([1,2,3])
manager = get_current_fig_manager()
manager.window.SetPosition((100,100))
#or: manager.window.Centre()
show()
Todd Miller offered a TkAgg example:
# set the window position to X=200, Y=300
get_current_fig_manager().window.wm_geometry("+200+300")
set xzeroaxis and set yzeroaxis,
use axhline() and axyline().
Prior to version 0.63,
Matplotlib does not have an equivalent of gnuplot's set xzeroaxis.
(This draws a horizontal line across the xrange at y=0.)
John Hunter explains how to achieve the same effect,
and it provides many insights into matplotlib.
"What you want to do is plot a line where the y coord is in data units and equal
to zero, and the x coord is in axes units and spans the xrange. The
line stretches from left to right regardless of the xlimits.
Each axes provides a data transform and an axes transform, so that you
can specify lines/text/etc in either coord system; axes coords are 0,0
for lower left and 1,1 for upper right. But what you want to do is
mix the data and axes transforms for x and y. The transforms module
provides a helper function for this. The following code is freestyle
(untested) but should work:
from matplotlib.transforms import blend_xy_sep_transform
ax = gca()
trans = blend_xy_sep_transform( ax.transAxes, ax.transData)
plot([0,1], [0,0], transform=trans)
pyplot.plot(d, f(d), color = (1,0,0)).
ax = pyplot.gca()
ax.get_xaxis().set_ticks_postion('bottom')
ax.get_yaxis().set_ticks_postion('left')
show only once in a script.
But sometimes you want to alternate seeing plots and
doing computations.
See http://stackoverflow.com/questions/458209/is-there-a-way-to-detach-matplotlib-plots-so-that-the-computation-can-continue
for a discussion.
Kawa offers a very useful collection of supplemental programs and data in EViews format, including the NBER historical series and the PENN World Tables.
EViews 5 can read and write many data formats.
Up to version 4,
the save command is not correctly documented in the EViews documentation.
If you provide a filename as an argument, it is really equivalent to File.SaveAs.
That is, it saves under the new name and changes the name of the current workfile.
In fact, as of version 4, this is the only way to change the name of a workfile in RAM.
In Eviews3, you could use scat(r) to get in a single graph
multiple scatter plots with their regression lines. This is not possible in
EViews4, where the replacement linefit command only uses the first
two members of its group argument.
missing observations: @obs(x[,s]) gives you the number of non-missing observations in x in the current sample or in sample "s" if the option is listed. One can access the number of non-missing observations for a group with @comobs.
%param1, equal to a complicated function of parameters, which recurs in a system estimation.
As a simple example:
%param1 = "c(1) + c(2)"
system1.add y = c(1)* x1 + x2/({%param1})
Oddly, string substitution only works for procs (commands) and not for assignments (using = ).
For example,
scalar trythis = ({%param1})
doesn't work.
(In this case, EViews will look for an object named c(1)+c(2) without evaluating the expression.)
Backwards compatibility: Eviews 3 and EViews 4 are not fully compatible. See the version 4 User's Guide page 532: "A Note on EViews Backward Compatibility" and page 580/581: "Converting from Version 3 Sspace" for some changes from version 3 to version 4. Briefly:
legend(s) becomes legend columns(1)
scat is not supported.
To reduce the image image resolution, use a video filter.
E.g.,
ffmpeg -i myinfinle.mp4 -vf scale=iw/2:ih/2 myoutfile.mp4
(Here iw and ih
hold the input resolution width and height.
See the filters documentation.)
To extract part of an MP4 file, copy the audio (acodec) and video (vcodec)
for a stime range. E.g.,
ffmpeg -ss 00:00:00 -to 00:10:00 -i myinfile.mp4 -acodec copy -vcodec copy myoutfile.mp4
extracts the first 10 minutes.
(See the documentation of time durations.)
You can pipe a series of frames (e.g., Matplotlib graphs) to FFMPEG to produce an animated graph.
You can create PNG files named (frame000.png, frame001.png, etc) and do
ffmpeg -b 8000 -f mpeg2video -r 30 -i frame%03d.png movie.mpeg.
You can also produce an AVI like this:
ffmpeg -r 10 -i frame%03d.png -f avi -vcodec msmpeg4v2 -y movie.avi.
Works great!
Note that AVI supports very low framerates; MPEG does not.
Vim is free and open source software; get it here: https://www.vim.org/. If you are new to Vim, get some documentation. Be sure to do the Vim tutorial and to read the help for "navigation". If you want additional documentation, consider Vi Improved—Vim by Steve Oualline or Vim ge-packt by Reinhard Wobst.
Windows installation hints:
Installation creats a vim directory.
Below that is a vimfiles directory.
For Windows users who are new to Vim,
i. copy my share_vimrc.vim
file to your vimfiles directory,
and then
ii.
look for the _vimrc file in your vim directory
and add at the bottom of that _vimrc file
the following command:
source $VIM/vimfiles/share_vimrc
Getting started:
Look for a file called vimtutor
or vimtutor.bat.
Set aside 20 minutes for the tutorial,
and then run the tutorial
(e.g., by clicking the file icon).
When you want to go deeper,
look at the Vimcasts videos.
The Vim cursor stays in the viewing window,
in contrast with some Windows applications.
If you want to cursor down and then return to where you started,
set a mark before moving. (See :h marks.)
But, if you want to be able to page down and then page back up to where you started,
use ctrl-d and ctrl-u rather than Pg-Dn and Pg-Up.
Digraphs allow simple and intuitive keyboard entry of unicode characters. Vim digraphs are based on RFC 1345. See ``:h digraph-table``.
To find the non-ASCII characters in a file,
use a regular expression.
To search for characters outside of the ASCII range,
search for [^\x00-\x7F].
(Alternatively, use decimal values: [^\d0-\d127].)
However to search for individual non-ASCII characters,
see the help for %o.
Colorscheme:
Vim provides excellent syntax highlighting in various color schemes.
There is an online gallery.
You can get a list of all the installed colorschemes by entering
colorscheme <ctrl+d>
at the Vim command line.
(Do not forget the space.)
I particularly like koehler, a rather dark color scheme.
(Download from GitHub.)
You can set this by entering colorscheme koehler at the Vim commmand line.
Another popular dark color scheme is solarized.
(Download from GitHub.)
To determine your current color scheme, enter echo g:colors_name
at the command line.
(If you did the tutorial, you know you can reach the command line
by pressing Esc
Buffers:
A buffer is a file open for editing.
Use the :ls command to list your current buffers.
This list is not interactive.
However, while looking at this list,
you can e.g. enter :b10 to switch to buffer 10.
Tabs:
Vim uses tabs in a unique way.
They are called tab pages,
and they are essentially an arrangement of windows open on some of your buffers.
(A buffer is a file that has been opened for editing;
a window is an area to view a buffer,
and a tab page is a collection of windows.
Vim has excellent syntax highlighting.
However, by default it does not use a lot of context.
In many settings, I find it useful to set
syn sych minlines=300.
When necessary, you can syn sych fromstart,
but this may cause a slowdown in very large files.
(See the help for syn-sync.
:help :python, :help python-examples,
and especially :help python-vim.
Start with
:python import vim :py cb = vim.current.bufferNote that cb is now an iterable container of strings. You can assign to the i-th line by assigning to cb[i].
Exiting a buffer: it is usually recommended to exit a buffer using bnext. However I sometimes prefer to delete a buffer as I leave it. Examining the help for ``bar``, we see that ``:w|bd`` can be used to write and then delete a buffer.
MiKTeX:
With minimal effort,
Vim cooperates wonderfully with LaTeX.
Windows users should install MiKTEX,
and then perhaps add my MiKTeX menus to the _gvimrc file.
Vim 7+ includes spell checking.
Spell checking is affected by syntax highlighting,
which identifies groups (e.g., comments).
For spell-checking longer LaTeX documents, I find I need to
setl spell syntax spell toplevel.
This ensures spellchecking of text that is not in a syntax item.
To make this permanent, you can include it in
~/.vim/after/syntax/tex.vim.
You can also the values of
syn sync maxlines
and
syn sync minlines
in the same file. (See the help for :syn-sync.)
If the file is not too big, you can even always start from the top with
syntax sync fromstart.
Learn to yank to registers.
For example, to yank (i.e., copy) the entire buffer you are working on
into the clipboard, use :%y+.
(The colon move to command mode,
the percent sign picks all lines,
the y is to yank,
and the plus sign picks the clipboard register.)
To remove the BOM from a file,
set nobomb.
There is a nice book devoted to learning Vim: Vi IMproved by Steve Oualline, also called the VIM book. You can view the Vim book online.
Be sure to try Jeff Lanzarotta's bufexplorer.vim. (One warning: as of early 2001, the buffer delete command in bufexplorer uses bd!, so changed buffers will be deleted upon request without a warning. Change this to bd.)
Thesaurus: You can adapt the Moby thesaurus for use with Vim.
Ctags is no longer included with Vim (it has grown into a project of its own.)
A useful plugin is Universal Text Linking, which executes URLs in plain text (e.g. call webbrowser, link MS Word documents, text files, embed vim commands).
Vim supports folding! You may find it useful to begin your writing with an outliner.
For user questions you can turn to the Vim mailing list. There is an archive.
Note that for all systems the Vim distribution is split into an archive with runtime files (documentation, syntax files, etc.) and a binary and/or source archive. You should download at least two archives!
You can find many useful hints at Softpanorama Western Orthodox Editors Page.
Count the occurences of a pattern with :%s/pattern//ng.
Count the number of words in a file with g ctrl-g.
To change line spacing see the help for linespace.
To change the font in gvim,
you can use the menu dialog Edit/SelectFont,
or you can include your selection in your _gvimrc file.
(I use set guifont=andale_mono:h9,courier_new:h10.)
Text editors should only use monospace fonts.
If you type very slowly, you need to slow down Vim's macro recognition. See the help for map-typing. I recommend resetting timeoutlen according to the example in the help for timeoutlen.
'. takes you to your last edit
If you want to fill an entire page with background color,
you do not need to know the dimension of the page.
Just start your PostScript program by setting the color
and then giving the command clippath fill.
(Of course few printers can print to the exact edge of the page.)
Transparency: PostScript uses an opaque imaging model. It does not support transparency. However, there is a workaround available if you have Adobe Distiller. You can add a "PDFMark" to your Postscript file to set the transparency for an object. When Adobe Distiller processes this PDFMark, it will set the transparency for the object. Lancaster offered an early discussion. (Start there rather than with the pdfmark Reference Manual.) Also, note that GhostScript supports transparency.
Use pdflatex to compile your documents.
Use tikz for your drawings.
To get double spacing, use \usepackage{setspace} \doublespacing.
This should leave your figures and tables alone.
For nice theorem and proof formatting,
consider combining the following three packages:
amsthm,
thmtools, and
thmbox.
(The last is just to redefine the proof environment.
If you use linerange markers with the listings package, make sure your markers do not include hyphens. (These will be treated as indicating the linerange break.)
The pict2e
package provides much needed additional functionality to the LaTeX picture environment.
As of June 2004, however, there is a small bug:
be sure to end qbezier and cbezier commands with an immediate comment character,
to avoid breaks in line continuity.
Using LaTeX to produce PDF files with graphics is pretty easy as long as you are producing clean graphics. But you need to pay attention to some details.
If you are producing your PDF files with dvipdfm, then you will find it easiest to use EPS graphics. (EPS is also the primary standard for publishers.) Historically this has been my preference, but lately I've been working more with pdflatex.
includegraphics
and putting the following in your preamble:
\ifx\pdfoutput\undefined
% we are running LaTeX, not pdflatex
\usepackage{graphicx}
\else
% we are running pdflatex, so convert .eps files to .pdf
\usepackage[pdftex]{graphicx}
\usepackage{epstopdf}
\fi
\documentclass{article}
\usepackage{graphicx}
\begin{document}
\begin{figure}[htp]
\centering
\includegraphics[width=\textwidth]{c:/temp.eps}
\caption{Test Figure}
\label{f:mytestfig}
\end{figure}
\end{document}
On the one hand it is easy enough to zip up all your EPS files with the embedding LaTeX document. On the other hand, one of the attractions of the LaTeX picture environment was that your drawings were present in your .tex file. Here is a way to get almost the same thing with EPS files. Add the moreverb package. Put your EPS file in a verbatimwrite environment in your .tex file. You will have to provide a file name that it will be written to, but once it is written you can read it into a figure as usual. (An alternatie is to generate your figures with embedded gnuplot commands, using the egplot package.)
\begin{wrapfigure}{R}{2in}
\includegraphics[width=2in]{example.eps}
\caption{Example Caption}
\label{fig:xmpl}
\end{wrapfigure}
name1+name2:date:journal
use the key name1.name2-date-journal.
name or id attribute
in an HTML or XML document.
The colon is allowed in HTML but is reserved in XML for namespace specification.
The plus-sign is not allowed in either specification.
XML allows a whole bunch on non-ASCII characters but HTML does not.
mankiw.etal-1994-qje)
so that your bib files remain useful to you years later.
month = jan,) so that months will end up correctly formatted.
(If you say, month="Jan.", it will always be formatted that way.)
Since gnuplot is a plotting package,
it offers limited data manipulation facilities.
These can be increased by the use of gawk,
which gnuplot can talk to nicely.
Alternatively you can manage your data in Python and use a Python interface to gnuplot,
such as Gnuplot.py.
If you use Jupyter’s IPython,
which is a good idea,
you can try the included enhanced version: Gnuplot2.py.
In either case,
if you use gnuplot's fit command,
you should consider how to get the results back into Python.
There are palettes (themes) for gnuplot.
Be sure to use the valuable documentation that gnuplot includes.
Begin by typing "help introduction" at the gnuplot prompt,
since this explains the syntax used in the help documents.
Note: if you download gnuplot for Windows as a ``.zip`` file,
the gnuplot help file (wgnuplot.chm) does not work properly.
Windows blocks it.
To unblock it, right click on the file,
pick ``Properties``, and then press ``Unblock``.
Please be sure you have an official release before doing this.
The command to load a script is load.
Provide your file name in quotes, using forward slashes.
Remember that gnuplot distinguishes integers and floats by the presence of a decimal point. Make sure your real numbers are recognized as real by including a decimal point. Integer division produces an integer result. For example, $3/2=1$ but $3./2=1.5$.
To easily plot a cobweb plot (i.e., Verhulst diagram with gnuplot, see this article.
Setting the color of plots is not well documented. Do it by setting the linetype variable. To see the linetypes, type test at the gnuplot command line.
To draw a 3-D scatterplot, try
splot 'data.asc' with points palette
You can draw a line segment by setting an arrow with the "nohead" parameter.
set log y 2.0 set format y '2^%L'
label command.
Use the 'front' option to make sure that they are placed after lines are plotted.
If you know the location,
you can also place a point as a data plot of a single value. E.g.,
plot '-' with points ps 3
1 1
e
centers a point at (1,1).
If you would like to use gnuplot to create a drawing with no plots in it this is possible.
Just be sure to set the yrange,
then plot 0/0 after all your drawing commands.
Up to version 3.8j, gnuplot arrowheads are not offset to account for their miter. This means they generally extend past the point where you expect them to stop. There is no satisfactory workaround, but you can try drawing a filled arrow with a very narrow pen and then redrawing just the tail (as a headless "arrow") with a thicker pen.
Kawano offers a brief overview of how to generate numerical plots using gnuplot.
I consider one extension to be indispensible: Session Manager. Another interesting extension is Taboo which allows you to replace the practice of keeping many tabs open with a single page of site links.
The default in Windows is to prevent you from seeing the extension to a filename. You can fix this as follows: open the Control Panel and open Folder Options, then pick the View tab and uncheck Hide File Extensions.
Windows 10 PowerToys
includes a Keyboard Manager,
for key remapping.
This is useful for mapping CapsLock to Ctrl.
SHOW SPLASH SCREEN=0You can pre-empt certain misconfigured servers that do not correctly handle RFC2616 add the following to the [Adv User Prefs] section of opera6.ini:
HTTP Accept Charset=windows-1252, utf-8, utf-16, iso-8859-1, iso-8859-15;q=0.6, *;q=0.1
NPSVG6.DLL SVG6RFF.DLL SVG6RR.DLLRestart Opera and test the plugin at the Adobe SVG test page. You should see 3 animated images.
New LaTeX users on Windows may wish to read these hints
export command.)
--include-directory option when you call pdflatex.
To include graphics in your document,
use the graphicx package.
Keep your images grouped together,
and point to them with the \graphicspath command.
(Some knowledgeable users suggest setting TEXINPUTS instead,
e.g., http://www.latex-project.org/cgi-bin/ltxbugs2html?pr=latex/2618.)
Note the odd syntax for \graphicspath:
each path must be in braces and must end with a forward slash.
If you want an images directory searched recursively
(say because you put each chapters images a separate folder),
end the path with a double slash.
E.g., \graphicspath{{images-eps//}{images-pdf//}}.
Most important: the TikZ/PGF manual is very helpful. Also, Crémer provides a useful introduction.
The intersection coordinate systems are very useful.
E.g., (A |- B) takes the first coordinate from A and the second coordinate from B.
See the PyX manual.
The arc command feels odd, especially to those with PostScript experience.
Instead of inputting a circle center, you must give the start point for the arc.
Suppose we want to draw an arc from from a circle centered at (1,1) with a radius of 2,
say from 30 degrees to 60 degrees.
You can do that like this:
\draw (1,1) +(30:2) arc (30:60:2)
Parametric plots are easy. E.g.,
\draw[domain=-20:0, smooth, variable=\t, samples=200] plot({cos(\t r)*exp(0.1*\t)}, {-sin(\t r)*exp(0.1*\t)});
To place text along a line, use the sloped keyword:
\draw (0,0) -- (5,1) node[above,very near end,sloped] {text}
You can use tikzset to set the default arrowhead style for your document:
\tikzset{>=latex}.
Predefined line thicknesses are:
ultra thin, very thin, thin (the default),
semithick, thick, very thick, ultra thick.
You can also set the line width explicitly:
\draw[line width=0.5pt].
A very light gray is black!10.
A very dark gray is black!90.
Use pgfplots for plots. The manual is good, and most tasks have very easy and obvious solutions. A few odd things are obscure. E.g., adding a zero-line to a plot.
When supplying images to accompany articles,
depending on the journal,
the TikZ format may not be acceptable.
Fortunately, it is easy to create standalone PDFs from TikZ figures.
Use the standalone documentclass.
% myfig.tex
\documentclass[tikz]{standalone} %standalone figure
\begin{document}
\begin{tikzpicture}
\draw (0,0) -| (5,5) -| (10,10); %tightly cropped!
\end{tikzpicture}
\end{document}
See the LuaTeX manual.
Recent TeX distributions include LuaTeX.
The first thing most new users will wish to attemp with LuaTeX
is compilation of legacy LaTeX documents. You can compile a LaTeX document named
mydoc.tex with the command lualatex mydoc.
To enter arbitrary Unicode characters on a Windows platform, see
http://www.fileformat.info/tip/microsoft/enter_unicode.htm.
On Linux, consider the IBus input bus.
If you use Vim on either platform, see help utf-8-typing (and help digraphs-default).
To see your recent files,
open the Run Dialog Box (keyboard shortcut Windows Key+R)
and enter shell:recent.
You can open (or delete) files here.
To search for a phrase in a given type of file in FileExplorer, use the search bar on the upper right hand corner with the type: and content: prefixes. E.g., content:"my phrase" type:.tex. (First, make sure you are indexing the contents.)
If you are using Subversion for version control,
make sure you set svn:eol-style to native
on your .lyx files.
To edit the LaTeX preamble, go to Document » Settings » LaTeX Preamble.
It is possible to provide a direct download link for items on your Google drive by making a small change in the URL. Here is an example for a Google spreadsheet. In a browser, open the file to edit and you'll see an address like https://docs.google.com/spreadsheets/d/ID/edit, where the ID will be specific to your document. Just change the end of this to specify export and a format argument. For example, https://docs.google.com/spreadsheets/d/ID/export?format=xlsx.
GnuWin32 includes many useful GNU applications,
including grep, sed, and awk.
If you download everything (recommended), it will take a while.
If you are just looking for a grep implementation,
you can try grepwin. (License: GPL)
There are more options. I do not use them and they can confuse some windows applications using the same name as those in the gnu package, so I will not explain. If you are interested, check the readme file.
Copyright © 1998 & 2020
by Alan G. Isaac
(mailto: a i s a a c @ a m e r i c a n . e d u)
Some Rights Reserved:
The content on this page is subject to the
Creative Commons Attribution License version 2.0