NetBeans IDE 6.8 with the C/C++/Fortran plugin features the I/O Usage tool for observing the
input/output behavior of C/C++ projects. The tool shows an overall graphical view of the reads and writes of your
project, and a more detailed view of individual files that are read from and written to, along with the number of bytes
that are read or written.
This tutorial demonstrates how to use the I/O Usage tool for C/C++ projects running on
Solaris 10 or OpenSolaris operating systems. The tool is supported only on Solaris operating systems
because the Solaris DTrace utility is required to gather the system data and program data for the tool.
The I/O Usage tool runs by default when you run your C or C++ NetBeans project on a Solaris operating system.
You can disable it if you want in your project properties.
If you are a Windows or MacOS X user and have a Solaris system available on your network, note that you
can run the IDE locally on your PC or Mac and set up your project to use the Solaris system as a remote development host.
Remote development would enable you to use the I/O Usage tool and other Solaris-based tools even when
you run the IDE in Windows or MacOS X.
The remote development feature is
only available in the C/C++/Fortran module for the NetBeans IDE.
See the
C/C++ Remote Development Tutorial for more information about how
to set up to build on a remote development host.
This tutorial uses a project running on a single-core Solaris SPARC processor-based system. You can follow the steps
of the tutorial if you run NetBeans IDE 6.8 on Solaris operating systems on SPARC-based or x86-based systems. Note that this
tutorial uses "Solaris operating systems" to mean both OpenSolaris and Solaris 10 operating systems.
I/O Data Collection
The input/output data is collected with DTrace, a tool for tracing the activities
of running programs in the Solaris 10 and OpenSolaris operating systems.
A special DTrace script intercepts I/O-related Solaris syscalls: open, creat, close, read,
write. This script aggregates the data and passes it to NetBeans IDE for
analysis, and the IDE then displays the information graphically, and in tabular form
when you click the I/O Details button.
Setting Up Your Solaris Environment for the Tutorial
In order to use the I/O Usage tool, the Solaris user account that you use when you run NetBeans must have sufficient DTrace privileges to observe system behavior.
The user account must have the dtrace_user, dtrace_proc, and dtrace_kernel privileges.
To check your DTrace privileges, type the following at the command prompt: /bin/ppriv $$
If your account has the required privileges, the ppriv command should return something similar to:
E: basic,dtrace_kernel,dtrace_proc,dtrace_user
I: basic,dtrace_kernel,dtrace_proc,dtrace_user
P: basic,dtrace_kernel,dtrace_proc,dtrace_user
L: all
The line that starts with "I:" is important because it specifies privileges that are inherited by programs
started from your shell. If your account doesn't have the required inheritable privileges, and you do not have administrator privileges or root access to your system, you should
ask your system administrator to add the dtrace_user, dtrace_proc, and dtrace_kernel inheritable privileges
to your account.
If you have administrator privileges or root access to your system, you can grant your user account the required privileges as described below.
To permanently grant required DTrace privileges to a user account:
Make sure the user account whose privileges you want to modify is logged out of the system.
Log in as superuser (root) or another administrator user.
Type the following at the command prompt, and replace username with the user account name you are modifying:
$ usermod -K defaultpriv=basic,dtrace_kernel,dtrace_user,dtrace_proc username
You can then log in to the user account, start the NetBeans IDE, and use the profiling tools with the DTrace data provider.
To temporarily grant required DTrace privileges to a user account:
Type the following to determine the process ID of the shell's process:
$ echo $$
Become superuser (root) or another administrator user.
Type the following, and replace process-ID with the process ID that was returned from the echo command :
$ ppriv -s I+dtrace_user,dtrace_proc,dtrace_kernelprocess-ID
All commands typed in the shell specified by process-ID now inherit the required privileges. The user account can start
the NetBeans IDE in this shell and use the I/O usage profiling tool.
Create the Project for the Tutorial
To explore the I/O Usage profiling features, we'll create a new project from
the ProfilingDemo sample application, which is included in the IDE.
The ProfilingDemo application has three stages in which it performs tasks repeatedly for 10
seconds, using three different techniques to generate some interesting activity to show in the profiling tools.
The first task is to write some data to a file, a variable number of subsequent tasks perform some
calculations. The number of calculation tasks depends on the number of CPUs.
For a single-CPU machine there are two tasks in the set: one writing to
a file, and one doing some calculations.
For a multi-CPU machine, the total number of tasks is equal to the number of CPUs.
Where N is the number of CPUs, there are N tasks in the set: one task writing to a file, and N-1 tasks
doing calculations.
For example, if a development host has 32 cores, the program performs one writing task, and 31 calculation tasks. You must press the Enter
key to begin each task, so you have to pay attention to the Output window.
The ProfilingDemo project's stages are as follows:
Sequential Demo is single-threaded and runs the two tasks, one after the other, each one for 10 seconds.
This can take some time on a system with a large number of CPUs because more sets of tasks are performed, one after another.
Parallel Demo is multithreaded and performs the same write to file and calculation tasks simultaneously in separate threads,
once for each CPU. This stage is much faster than the Sequential Demo on multicore machines because the same
number of tasks are performed as in the Sequential Demo, but they are executed simultaneously. However, the threads are not
prevented from competively accessing the same data, which can lead to inaccurate results. The threads in the ProfilingDemo
do not have shared data, so there is no risk of getting inaccurate results. But of course in other programs there is no such guarantee.
Pthread Mutex Demo is also multithreaded and performs the same write task and calculation tasks as the other demo stages. However,
the code uses mutual exclusion locks to prevent overlapping access to certain functions by multiple threads.
To create the ProfilingDemo Project
Open the New Project wizard by choosing File > New Project.
In the wizard, select the Samples category, and then the C/C++ subcategory.
Select the Profiling Demo sample as shown in the figure and click Next.
You can choose the name of the project and the location of the project. Here we
use the default ProfilingDemo_1 in our NetBeansProjects directory.
Click Finish to exit the wizard and create the project.
Set the Project's Profile Configuration
The project properties includes a profile configuration to enable you to set which of the profiling tools you
want to run when you run your project. To see the I/O Usage tool, you need to use the C/C++ DTrace
Extended profile configuration, which specifies all the available profiling tools. However, for this
tutorial we want to focus on the I/O Usage tool, so we will make our own profile configuration to select a smaller
set of profiling tools. This task is optional.
Do the following to create a new profile configuration:
Right click the project name and select Properties to open the Project Properties dialog box.
Select the Profile category. The C/C++ DTrace Extended profile configuration is selected.
Click the button labelled "..." next to the Profile Configuration list to open the Profile Tools Manager dialog box.
From the Profiler Configurations list, select Manage Configurations...
In the Profiler Configurations Manager dialog box, select C/C++ DTrace Extended.
Click Duplicate.
Click Rename and type CPU Memory I/O, then click OK.
This screenshot shows the Name dialog and the prior dialog boxes you need to access to get to it.
Click OK in the Profile Configuration Manager dialog box.
Select the new CPU Memory I/O configuration in Profile Tools Manager dialog box.
Deselect the Thread Microstates and Thread Usage tools in the list of tools and click OK.
To use the configuration, in the Project Properties dialog box, select CPU Memory I/O for
the Profile Configuration, and click OK.
Build and Run the ProfilingDemo Project
(Optional) If you want the output of the project to be displayed in the Output Window in NetBeans as shown in this tutorial,
right-click the ProfilingDemo project node and select Properties. In the Project Properties,
click the Run node in the Categories panel, and for Console Type select Output Window. Click OK.
Right-click the ProfilingDemo project node and select Build.
The Output tab shows the results of the build, similar to that shown below.
The compiler shown in the screenshot is the Sun Studio tool collection's
cc compiler. You can also use the GNU C compiler, gcc.
Right-click the ProfilingDemo project node and select Run.
Press Enter whenever you are prompted during the run of the ProfilingDemo project.
The Run Monitor tab opens to display the dynamic graph for I/O Usage tool,
and any other tools that are defined for the profile configuration. Here we are
using the CPU Memory I/O profile configuration that we created, so we only see
those three tools.
If you want to see all the tools, you might need to increase the size of the Run Monitor tab by clicking and dragging
the top border of the Run Monitor tab upwards. In the screenshot above, the Run Monitor tab has been enlarged both
vertically and horizontally to make the indicators easier to show. You can also increase the height of a particular graph by
dragging the top border of the tool.
Notice in the output window the ProfilingDemo program tells you what it is doing
so you can match it to the data that the IDE is representing graphically in the tools. For example,
the program displays the elapsed run time before it asks you to press Enter, and displays
how much memory it is allocating, performs calculations, and
then frees the memory. You can see the graphs reflect the program's activity.
For the purposes of this tutorial, please run the program through to completion, pressing Enter whenever prompted. The data
gathered will be saved and we can take our time to step through it and explore the I/O Usage tool's features.
Explore I/O Usage
The I/O Usage tool
shows an overview of the program's read and write activity during the project's run.
The screenshot below was taken at the beginning of the program's run, during the SEQUENTIAL DEMO portion, in which two tasks
are run one after the other in a single thread. The first few seconds the program starts, and then is waiting for the user to press Enter.
When the user presses Enter at about 0:02 seconds, the program writes characters to a temporary file. This activity is reflected in the
I/O Usage tool as the orange line showing bytes written. Notice the following:
I/O Usage tool's orange line shows the number of bytes written in the last second. The ProfilingDemo itself reports
that it writes a total of 39690240 bytes or about 38.7M during this run of the sequential demo. If you added up all the data points on
the orange line from 2 seconds to 12 seconds, the total would be close to 38.7M bytes.
CPU Usage tool's System time is high during this phase because the program makes use of system calls to generate data
and write it to disk.
Memory Usage tool shows a steady 8K bytes of memory heap allocated.
At the bottom of the Run Monitor window, you can see sliders for controlling your view of the graphs.
Place your mouse cursor over the end points for information about them.
Click and hold the mouse button on the horizontal scroll bar located at the bottom, and drag the slider
to the left to see the beginning of the run. All the graphs slide in unison so you can
see what happened in each area (CPU, memory, threads) at any given time, and see the relationships
between them.
Drag from left to right to see the complete run.
Click the I/O Details button. The I/O Usage details tab opens, displaying standard input, standard output, and temporary files that
the program reads from and writes to.
The files with checkmarks have been closed, and the files that are marked with a yellow icon are still opened for read/write actions.
Notice that this program only requires input of the Enter key from time to time, so the Bytes Read column is not very useful.
You can get rid of the Bytes Read column by clicking the button to the right of the column headings to bring up a Change Visible Columns
dialog box. Click the checkbox to deselect Bytes Read and click OK.
Suppose you want more details about how this program uses all these temporary files.
Click one of the filenames in the I/O Details tab to see the functions that have opened and closed the file.
The function names are
displayed in the panel to the right of the file table.
Double-click a function in the right panel to open the source file where the function is called.
In this case, we clicked the
work_run_syscpu function, and the common.c file opened.
In the I/O Details tab, you can specify a time interval to look at: select or type the number of seconds for the Start and End times of
the interval you want in the two time selectors. If you type the seconds, you might need to press Enter.
This updates the list of functions to show only the data that was gathered during
that time interval so you can see exactly how many bytes were written and read to which files during any particular
time. The graph also updates
to show the selected time interval in bright contrast to the rest of the run time, which is darkened.
In the Run Monitor window, we see write activity began very near the beginning of the run. This is the point
where the sequential demo portion of the profiling project begins writing to a temporary file. The sequential demo
had two tasks of 10 seconds each, so lets guess that to isolate the sequential demo activity, we should look at
0:00 to 0:27 seconds of the run to account for delays in pressing Enter and so on.
Select 0:27 for the End time in the I/O Details tab.
The data is then filtered so we can focus on the input/output of the sequential phase. Notice that
the graphic indicators also show the effect of selecting the segment of time to filter.
Notice that there is one temporary file shown in the I/O Details tab during the sequential demo interval. A single thread opens
and writes to the file, then closes it. Click on the file to see the functions that access the file, and double-click a function
to see the source code.
Back in the Run Monitor window, we scroll the time slider to about 0:30 seconds, where we see write activity begin again after a pause. The
pause in writing occurs during the sequential demo's calculation task, in which no disk writes occur. The renewed activity shows
that the program is entering the PARALLEL DEMO portion and the user presses Enter to launch the tasks, where the write to disk and the calculation
take place simultneously in separate threads. To isolate the parallel demo activity, in the I/O Details tab's Time Filer, enter
0:28 to Start and 0:41 seconds to End and press Enter to filter the data to focus on the input/output of the parallel phase.
Notice that there are several temporary files shown in the I/O Details tab during the parallel demo interval.
This occurs even though there is only one thread writing to a file because each second it switches to a new file. The
calculating task does not write to disk.
Click one of the temporary files and see that the parallel_threadfunc function is opening and closing files.
In the screenshot below at about 0:40 seconds, the I/O activity changes again. At this point, the
program enters the PTHREAD MUTEX DEMO portion, and the user presses Enter.
Filter data for this area by selecting 0:41 for the Start time and max for the End. Press Enter is
you type the values in.
The I/O Details tab shows multiple files are open during the PTHREAD MUTEX DEMO interval.
One thread is writing to a file, and every second it switches the file it writes to, as in the Parallel Demo interval. However,
because mutex locks are used, sometimes the writing thread is blocked by the calculating thread and cannot continuously
write to files.
Try running this program again with the Thread Usage tool enabled. You can add it to the profile configuration
that we made in this tutorial, by selecting Tools > Profile Tools, then selecting the configuration and
selecting the Thread Usage tool.
See Also
For more information about the DTrace technology that is used behind the scenes to compile data about the running project,
see the BigAdmin DTrace Portal.
This tutorial demonstrated the following points about the I/O Usage profiling tool:
I/O information is collected on Solaris platforms automatically with the Solaris DTrace utility.
Graphical profiling indicators that are displayed can be selected through the use of Profile Configurations.
The I/O Usage tool can only be found in the C/C++ DTrace Extended profile configuration or your own configuration.
Profiling settings are located in project properties, in the Profile category.
Clicking the I/O Details button causes detailed information about the I/O by the program's functions to display.
Clicking functions in the detail tab causes the source file containing the function to be opened at the line number where the function is called
If you have access to a system running a Solaris operating system on your network, you can use the I/O Usage tool
from your Windows or Mac machine by setting up the Solaris system as a remote development host for building and running
your projects.
