Using ANTS Memory Profiler to track down a memory leak in a WinForms application

Source: http:www.red-gate.com/products

Using ANTS Memory Profiler to track down a memory leak in a WinForms application
This walkthrough shows how to locate a memory leak using a sample application called QueryBee. QueryBee is a simple WinForms application for running queries against SQL Server databases. It has a database connection dialog and a query window to query the database. We know our application is leaking memory, because every time we open a query window and close it again, our memory usage increases.

When you open the profiler, you first see the Startup screen:



Figure 1. The ANTS Memory Profiler startup screen.

Here, there’s a list of your recent profiling sessions so you can re-run them easily. For this example, we’ll start a new session by clicking New profiling session.

The New profiling session screen is displayed:



Figure 2. It's easy to configure and start a new profiling session.

All we need to do is point it at QueryBee, choose our performance counters, and click Start profiling.

The profiler starts up QueryBee and begins collecting performance counter data:



Figure 3. Whilst profiling, ANTS Memory Profiler collects performance counter data. The profiler is telling us that it's profiling our application. There are also some useful instructions on this screen telling us to take and compare snapshots.

Taking and comparing memory snapshots is a key activity when looking for memory leaks, so our approach will be as follows:
Wait for QueryBee to open.
Take a first snapshot without using the application; this first snapshot will be used as a baseline.
Within QueryBee, perform the actions that we think cause the memory leak.
Take a second snapshot.
Examine the comparison that the profiler shows us after it has finished taking and analyzing the second snapshot.

So, QueryBee is open, sitting in our system tray.

At this point, we take a first snapshot, which we will use as a baseline for comparison with later snapshots.

When we click the Take Memory Snapshot button, the memory profiler forces a full garbage collection and takes a snapshot of the heap memory it is using.



Figure 4. Results from our first snapshot – Summary screen.

Now, we go back to QueryBee and perform the tasks which we think cause the memory leak.

We open up QueryBee and connect to a database.



Figure 5. QueryBee – Database connection dialog.



Figure 6. QueryBee – The query window.

The query window opens up and we enter and execute a SQL query.

We obtain some results and close the query window.



Figure 7. QueryBee – The results are displayed in a grid.

We close the query form.

At this point, the window is gone. We expect the memory usage to fall back to where it was in the first snapshot, but that is not the case.



Figure 8. Despite closing our query window, the memory usage has not fallen.

So what's happening here? We take a second snapshot and get the results.



Figure 9. The summary pane compares the results of the two snapshots.

A number of problems are highlighted by the summary screen.
We can see a large memory increase between snapshots, which we noticed on the timeline (top left).
The Large Object Heap appears to be fragmented, which could cause problems (top right).
The Generation 2 heap accounts for a large proportion of memory usage - often indicating objects are being held onto for longer than necessary (bottom left).

We can choose to select one of the largest classes which are shown to us in the bottom right of the screen, but instead we switch to the class list to find out more. The class list gives us a fuller picture of what's in the snapshot.

We're interested in objects which have been created since the baseline snapshot, so we need to look at types which have more instances in the second snapshot. We therefore sort by Instance Diff in decreasing order.



Figure 10. The class list allows you to compare memory usage in both snapshots in more detail.

The String class has been placed at the top of the list, with over 300,000 new instances. We want to understand why there is such a large increase so load the Instance Categorizer for the String class by clicking the  icon.



Figure 11. The Instance Categorizer shows chains of instances sorted by their shortest path to GC Root.

We see that over 21MB of the String class are held in memory by the same shortest path back to GC Root, via our QueryForm and ConnectForm. We select Show the Instances on this Path to view a list of every instance in the previous category.



Figure 12. The instance list view shows us a set of strings which we recognize as coming from our SQL Database.

The Instance List is showing us data which QueryBee had retrieved from the SQL Database, but that data should have been destroyed when QueryForm was closed. We select one of the instances and click the  icon to generate an Instance Retention Graph.



Figure 13. This instance retention graph.

Using the instance retention graph, we should be able to find out what is still referencing our String instances. Then, we'll be able to go back into our code to break the chain of references that is keeping them in memory.

We start at the bottom and work our way up the graph until we find a reference that needs to be broken. We'll just need to break the chain at one point to allow the garbage collector to clean up everything below that.

By navigating up, we can see the string is being held onto by QueryForm, even though that should have been released from memory. Looking a little further up, the graph is telling us that a System.EventHandler is referencing QueryForm and, if we step up one more level, it's telling us that the event handler is referenced by our ConnectForm instance – this is the form that asked us for the database connection details. In other words, the ConnectForm is holding onto the QueryForm via an Event Handler.

If we look at this node more closely, we see that it's actually being referenced by the ConnectForm's Foregrounded field.

Let's find this Foregrounded event in our code. We right-click on the QueryBee.ConnectForm node and open the ConnectFormsource code in Visual Studio™.



Figure 14. Foregrounded event in the ConnectForm source code.

The profiler automatically jumps to the Foregrounded event. We check where it is being used by right-clicking on Find All References.



Figure 15. The Foregrounded event is used in three places.

We've got three usages and we find that the last usage is where QueryForm registers for the Foregrounded event, but it doesn't look like it unregisters. If we fix that, then the memory leak should go away.

Once we're done, we need to rebuild, but first we need to stop profiling QueryBee so that the executable isn't locked. We go back to Profiler and click on the Stop Profiling button.

Then, we rebuild.



Figure 16. We rebuild our QueryBee application.

Back in the profiler, we start up a new profiling session. We want to find out whether the reference to the QueryForm has disappeared.

Note that it remembered our settings from last time, so all we need to do is click Start Profiling.



Figure 17. The settings dialog remembers settings from last time.

We take a first snapshot to use as a baseline.

We perform the same actions as last time: take a baseline snapshot while QueryBee is idle, then a snapshot once we’ve connected and run a query.

We'll also take an extra snapshot, because we want to be able to verify that the QueryForm has disappeared.

Finally, we close the query window with the results grid and we take a third snapshot.

We switch to a comparison between snapshots 1 and 3, using the snapshot selection field just under the timeline.



Figure 18. Summary screen comparing snapshots 1 and 3.

We can see there is now only a small memory increase between the snapshots, which is promising. Let's see if there's a QueryForm still in the class list.

We switch to the class list view and search only for classes in the QueryBee namespace.



Figure 19. Class list for the QueryBee namespace.

No, it's gone. We're no longer leaking the form

ANTS Performance Profiler Tool

Source: http://www.red-gate.com/products

It is a Profile and boost the performance of your .NET applications

ANTS Performance Profiler is an application profiler for .NET desktop, ASP.NET, and ASP.NET MVC applications:
Find performance bottlenecks fast by profiling both the .NET code and the data access layer
Get rich performance data, right-down to line-level timings and expensive database queries
Save time going round in circles diagnosing and debugging – let the profiler do the hard work for you
Explore unfamiliar code bases
NEW! Enhanced data access profiling: added support for profiling MySQL (or MariaDB) and PostgreSQL, so you can see queries run against those databases too

Drill down to slow lines of code with line-level timings
Profile C# or any other .NET code line by line, with precise timing data so you can find issues at a glance. Expensive lines of code are automatically highlighted for quick visual inspection.

Jump straight to the slowest activity
The call tree shows performance data for every method and identifies the most expensive methods, database queries, and web requests.

Capture outgoing web requests *
Look at your application's outgoing http requests. Get data about the request and response headers, and see what .NET code caused them to run.

Look at your code's database interaction *
Understand how your .NET code makes database queries and how those queries perform. ANTS Performance Profiler supports SQL Server and Oracle databases, whether local or remote.

Immediate feedback on application performance
Use the interactive timeline to check the CPU usage of your .NET or ASP.NET application and highlight problem areas to focus only on the data that matters.
Group methods by HTTP request *

Code and database activity are grouped by HTTP request, exposing performance problems on specific web pages.

Profile async code
Understand the performance of C#5 async code. View activity in the context of the code you wrote, and see how work being done asynchronously is related to the async method which caused it to run.

Decompile third-party code *
Find bottlenecks in 3rd party components and framework assemblies using integrated decompilation, powered by .NET Reflector.

.NET developers use ANTS Memory Profiler to:Find memory leaks within minutes
Optimize the memory usage of your C# and VB.NET code
Profile your code's use of unmanaged memory
Create better performing, less resource-intensive applications
Available as a standalone tool or as part of our .NET Developer Bundle
Why ANTS Memory Profiler?


Automatic memory management in .NET makes development a lot easier, but it's still easy to introduce memory leaks into your application. For example, forgetting to unregister event handlers can lead to significant chunks of memory being held on to unnecessarily, but spotting that this is happening can be very difficult.

Trying to understand memory problems without a suitable tool can take hours or days, if you're lucky enough to know a problem exists. This is time which would be better spent fixing the problem.

A tool every .NET developer should use
ANTS Memory Profiler makes memory profiling simple. With it, you can:
Obtain clear, meaningful results, making it easier for you to interpret the information. Spend your time fixing problems instead of struggling to understand them.
Get results fast – with a footprint of less than 32 MB, the profiler can comfortably profile large, complex applications, with virtually no overhead. Take as many snapshots of the heap memory as you like, in seconds rather than minutes.
Quickly visualize the relationship between your objects – use the instance retention graph to quickly see why your leaking objects are still being held in memory. You don't have to build a mental map to keep track of how objects reference each other.
Go straight to the source of the problem - intelligent analysis highlights the most likely causes of issues, often saving hours of problem-hunting.
Zero in fast on the causes of memory leaks – powerful filtering options allow you to cut through the noise, enabling you to quickly get to the root of even the most complex problems.

Disk cleanup in windows server 2003 (From command line)

In some times our system may go down due to the unnecessary files or recycled files or setup files or compressed zip files on our system.

In windows 2003 server we do not have an option  to clean the disk same as windows 7

In Windows Server 2003:

1.Go to Start 
2.Type RUN ,click on RUN icon
3.Enter cleanmgr at the command line then it will display the below screen




Check the boxes which are to be deleted,then click on OK.




In Windows 7:

1.Go to Start -All Programs - Accessories - System Tools - Disk Cleanu
p

Perfmon to capture the Process Performance of server or system

 How do we track and log system and process information on a Windows operating system?

 Windows system tools available for monitoring almost every type of performance, including CPU, memory, file system and network usage. You do not have to rely merely on the Task Manager.

There is a Windows utility called PerfMon (Performance Monitor) that has the ability to graph and log performance metrics for specific processes, as well as set alarms and timing for performance monitoring and logging.

Steps to View and Log Performance Data Using Windows 7 Performance Monitor:

1.Make sure LabVIEW is open.

2.Click on the Start Menu and click Run.
3.Type perfmon into the Run command prompt and click OK.



4.You will then see the Performance Monitor pop up.



5.Click on the green ‘+’ sign near the top of the Performance Monitor Window to bring up theAdd Counters window.

6.On the left side,choose which counters to add and click Add >>. Your counters should now appear under Added Counters. Click OK.



There are many options for what counters to add. A few that may be of special note when dealing with LabVIEW performance issues are:
Memory
Processor
Process (shown above): Under process, one can choose specific programs to monitor. In the upper left Window, you can select the aspects of the process that you’d like to monitor such as % Processor Time or Virtual Bytes. In the lower left hand window, you can select which process you’d like to monitor. Above, LabVIEW has been selected.

Now you should be able to see all chosen counters updating on the graph

In order to log this data to file, right-click on Performance Monitor and select New»Data Collector Set.


Type in a name for your data set and press Next.



Complete the rest of the steps including choosing the location you want to save your log file.
When you want to start logging the performance data, right-click on your Data Collector Set and select Start. To stop logging, right-click and select Stop.





Note: PerfMon provides ActiveX properties and methods, allowing you to control it through another ADE and even use it as an embedded control in an application.

Bugzilla-A Bug tracking tool

What is Bugzilla?
Bugzilla is a bug tracking system developed at mozilla.org.

How do enter a bug in Bugzilla?
To enter a bug, through "Enter a new bug" link from the main Bugzilla page. This will take you to a product selection screen.

What happens once enter a bug?
After you enter a bug, mail is sent both to you and the QA department. A member of the QA department will verify that they can reproduce your bug.

How do search a bug?
To search a bug, through "Query" link from the main Bugzilla page.

How do submit a patch?
The new Bugzilla system supports the attachment of patches, test cases, and various other forms of file types directly from the bug report screen. Just click on "Create an attachment"

Are cookies required in Bugzilla?
Yes.

How can you view your assigned bugs?
We can view the assigned bugs though “My Bugs” link.

How can you generate bug report?
We can generate bug report through “Report” link.

When Bugzilla was released?
Bugzilla was released in 1998.

Which language was written Bugzilla first time?
Bugzilla was originally written in "TCL".

Who developed the Bugzilla?
Terry Weissman

Which language Bugzilla written?
"Perl"

What are the Bugzilla fields?
Bugzilla have 11 fields like-
  1. Product
  2. Component
  3. Version
  4. Platform
  5. OS
  6. Priority
  7. Severity
  8. Assigned To
  9. URL
  10. Summary
  11. Description

How can you edit your account in Bugzilla?
We can use "User Preferences" link.

How can you add new Product in Bugzilla?
We can use "Product" link for adding new product.

How can you add Components of product in Bugzilla?
We can use "Components" link for adding components in Bugzilla.

How can edit version of any product in Bugzilla?
We can use "Edit Versions" link.

SCOM (System Center Operations Manager) Monitoring tool

System Center Operations Manager 2012 – the complete application monitoring solution

For many years Operations Manager has delivered infrastructure monitoring, providing a strong foundation on which we can build to deliver application performance monitoring. It is important to understand that in order to provide the application level performance monitoring, we must first have a solid infrastructure monitoring solution in place. After all, if an application is having a performance issue, we must first establish if the issue is due to an underlying platform problem, or within the application itself.

A key value that Operations Manager 2012 delivers is a solution that uses the same tools to monitor with visibility across infrastructure AND applications.

To deliver application performance monitoring, we provide 4 key capabilities in Operations Manager 2012:
Infrastructure monitoring – network, hardware and operating system
Server-side application monitoring – monitoring the actual code that is executed and delivered by the application
Client-side application monitoring – end-user experiences related to page load times, server and network latency, and client-side scripting exceptions
Synthetic transaction – pre-recorded testing paths through the application that highlight availability, response times, and unexpected responses

Configuring application performance monitoring

So it must be hard to configure all this right? Lots of things to know, application domain knowledge, settings, configurations? Rest assured, this is not the case! We make it incredibly easy to enable application performance monitoring!

1. Define the application to monitor.



2. Configure server-side monitoring to be enabled and set your performance thresholds



3. Configure client-side monitoring to be enabled and set your performance thresholds



And that’s it, you’re now set to go. Of course setting the threshold levels is the most important part of this, and that is the one thing we can’t do for you… you know your application and what the acceptable performance level is.
Configuring an application performance dashboard in 4 steps

It’s great that we make the configuration of application performance monitoring so easy, but making that information available in a concise, impactful manner is just as important.

We have worked hard to make the creation of dashboards incredibly easy, with a wizard driven experience. You can create an application level dashboard in just 4 steps:

1. Choose where to store the dashboard



2. Choose your layout structure. There are many different layouts available.



3. Specify which information you want to be part of your dashboard.



4. Choose who has access to the dashboard. As you will see a little later in this article, publishing information through web and SharePoint portals is very easy.



And just like that, you’ve created and published an application performance monitoring dashboard!


Anyone who has either worked in IT, or been the owner of an application knows the conversations and finger pointing that can go on when users complain about poor performance. Is it the hardware, the platform, a code issue or a network problem?

This is where the complete solution from Operations Manager 2012 really provides an incredible solution. It’s great that an application and associated resources are highly available, but availability does not equal performance. Indeed, an application can be highly available (the ‘5 nines’) but performing below required performance thresholds.

The diagram below shows an application dashboard that I created using the 4 steps above for a sample application. You can see that the application is available and ‘green’ across the board. But the end users are having performance issues. This is highlighted by the client side alerts about performance.






Deep Insight into application performance

Once you know that there is an issue, Operations Manager 2012 provides the ability to drill into the alert down to the code level to see exactly what is going on and where the issue is.




Reporting and trending analysis

An important aspect of application performance monitoring is to be able to see how your applications are performing over time, and to be able to quickly gain visibility into common issues and problematic components of the application.

In the report shown below, you can see that we can quickly see areas of the application we need to focus on, and also understand how these components are related to other parts of the application, and may be causing flow-on effects.




Easily make information available
With Operations Manager 2012, we have made it very easy to delegate and publish information across multiple content access solutions. Operations staff have access to the Operations Manager console, and we can now easily publish delegated information to the Silverlight based Operations web console and also to SharePoint webparts

lr_paramarr_random function in load runner

In performance testing, it is really important to simulate a realistic user path through an application. For example, randomly select an image link from a gallery or select a share from a share list. In such situations, you can use the LoadRunner lr_paramarr_randomfunction to select a random value from a captured parameter array. Similarly, you can also write a code to do the same.

Before you use the above function, you will need to use web_reg_save_paramfunction to capture all the ordinal values. This can be achieved by passing "ORD=ALL" into the function.

The following code demonstrates the use of lr_paramarr_random function. The code saves link Ids using web_reg_save_param function and then uses


Example:

This example shows how to get a random value from a parameter array.

char * FlightVal;

web_reg_save_param("outFlightVal",

"LB=outboundFlight value=", "RB=>",

"ORD=ALL",

"SaveLen=18",

LAST );

web_submit_form("reservations.pl",

"Snapshot=t4.inf",

ITEMDATA,

"Name=depart", "Value=London", ENDITEM,

"Name=departDate", "Value=11/20/2003", ENDITEM,

"Name=arrive", "Value=New York", ENDITEM,

"Name=returnDate", "Value=11/21/2003", ENDITEM,

"Name=numPassengers", "Value=1", ENDITEM,

"Name=roundtrip", "Value=", ENDITEM,

"Name=seatPref", "Value=None", ENDITEM,

"Name=seatType", "Value=Coach", ENDITEM,

"Name=findFlights.x", "Value=83", ENDITEM,

"Name=findFlights.y", "Value=16", ENDITEM,

LAST );

/*

The result of the web_reg_save_param having been called before the web_submit_form is:

Notify: Saving Parameter "outFlightVal_1 = 230;378;11/20/2003"

Notify: Saving Parameter "outFlightVal_2 = 231;337;11/20/2003"

Notify: Saving Parameter "outFlightVal_3 = 232;357;11/20/2003"

Notify: Saving Parameter "outFlightVal_4 = 233;309;11/20/2003"

Notify: Saving Parameter "outFlightVal_count = 4"

*/

FlightVal = lr_paramarr_random("outFlightVal");

Base64 Encode/Decode for LoadRunner

Code:

#include "base64.h"
vuser_init()
{
int res;
// ENCODE
lr_save_string("ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789","plain");
b64_encode_string( lr_eval_string("{plain}"), "b64str" );
lr_output_message("Encoded: %s", lr_eval_string("{b64str}") );

// DECODE
b64_decode_string( lr_eval_string("{b64str}"), "plain2" );
lr_output_message("Decoded: %s", lr_eval_string("{plain2}") );

// Verify decoded matches original plain text
res = strcmp( lr_eval_string("{plain}"), lr_eval_string("{plain2}") );
if (res==0) lr_output_message("Decoded matches original plain text");

return 0;
}

 base64.h include file

/*
Base 64 Encode and Decode functions for LoadRunner
==================================================
This include file provides functions to Encode and Decode
LoadRunner variables. It's based on source codes found on the
internet and has been modified to work in LoadRunner.

Created by Kim Sandell / Celarius - www.celarius.com
*/
// Encoding lookup table
char base64encode_lut[] = {
'A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q',
'R','S','T','U','V','W','X','Y','Z','a','b','c','d','e','f','g','h',
'i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y',
'z','0','1','2','3','4','5','6','7','8','9','+','/','='};

// Decode lookup table
char base64decode_lut[] = {
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0,62, 0, 0, 0,63,52,53,54,55,56,57,58,59,60,61, 0, 0,
0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,10,11,12,13,14,
15,16,17,18,19,20,21,22,23,24,25, 0, 0, 0, 0, 0, 0,26,27,28,
29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,
49,50,51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, };

void base64encode(char *src, char *dest, int len)
// Encodes a buffer to base64
{
int i=0, slen=strlen(src);
for(i=0;i
{ // Enc next 4 characters
*(dest++)=base64encode_lut[(*src&0xFC)>>0x2];
*(dest++)=base64encode_lut[(*src&0x3)<<0x4 amp="" src="" xf0="">>0x4];
*(dest++)=((i+1)>0x6]:'=';
*(dest++)=((i+2)
}
*dest='\0'; // Append terminator
}

void base64decode(char *src, char *dest, int len)
// Encodes a buffer to base64
{
int i=0, slen=strlen(src);
for(i=0;i
{ // Store next 4 chars in vars for faster access
char c1=base64decode_lut[*src], c2=base64decode_lut[*(src+1)], c3=base64decode_lut[*(src+2)], c4=base64decode_lut[*(src+3)];
// Decode to 3 chars
*(dest++)=(c1&0x3F)<<0x2 amp="" c2="" x30="">>0x4;
*(dest++)=(c3!=64)?((c2&0xF)<<0x4 amp="" c3="" x3c="">>0x2):'\0';
*(dest++)=(c4!=64)?((c3&0x3)<<0x6 amp="" c4="" div="" x3f="">
}
*dest='\0'; // Append terminator
}

int b64_encode_string( char *source, char *lrvar )
// ----------------------------------------------------------------------------
// Encodes a string to base64 format
//
// Parameters:
// source Pointer to source string to encode
// lrvar LR variable where base64 encoded string is stored
//
// Example:
//
// b64_encode_string( "Encode Me!", "b64" )
// ----------------------------------------------------------------------------
{
int dest_size;
int res;
char *dest;
// Allocate dest buffer
dest_size = 1 + ((strlen(source)+2)/3*4);
dest = (char *)malloc(dest_size);
memset(dest,0,dest_size);
// Encode & Save
base64encode(source, dest, dest_size);
lr_save_string( dest, lrvar );
// Free dest buffer
res = strlen(dest);
free(dest);
// Return length of dest string
return res;
}

int b64_decode_string( char *source, char *lrvar )
// ----------------------------------------------------------------------------
// Decodes a base64 string to plaintext
//
// Parameters:
// source Pointer to source base64 encoded string
// lrvar LR variable where decoded string is stored
//
// Example:
//
// b64_decode_string( lr_eval_string("{b64}"), "Plain" )
// ----------------------------------------------------------------------------
{
int dest_size;
int res;
char *dest;
// Allocate dest buffer
dest_size = strlen(source);
dest = (char *)malloc(dest_size);
memset(dest,0,dest_size);
// Encode & Save
base64decode(source, dest, dest_size);
lr_save_string( dest, lrvar );
// Free dest buffer
res = strlen(dest);
free(dest);
// Return length of dest string
return res;

Oracle Goldengate Technology

After Oracle corp. acquiring Goldengate software there is a lot of buzz about Oracle Goldengate and it is one of the hot topics at Oracle open world 2010.Oracle Goldengate can be used as a replication tool, ETL, and even as a DR solution.

Oracle Goldengate (Golden Gate) is probably the best replication software and it is very easy to configure and deploy it in large scale environment. Here are some of the things you need to be aware of:
  •  All Golden Gate configuration files are ascii text based files. Very easy to make changes but it is prone to human errors in an environment having many DBA's working on it.
  •  In order to use parallel apply threads, Golden Gate breaks down the database transaction into multiple transactions based on the hashing key defined for range split of the data. So, transactional consistency will not be guaranteed during real time but there won't be any data loss, but make sure that your application can tolerate this.
  • If there is no primary key or unique index exists on any table, Golden Gate will use all the columns as supplemental logging key pair for both extracts and replicats. But if you define key columns in the Golden Gate extract parameter file and if you don't have the supplemental logging enabled on that key columns combination, then Golden Gate will assume missing key columns record data as "NULL", which is a huge deal, and this will introduce logical data corruption on the target.
  •  Golden Gate started supporting bulk data loads with their 11.1 release but any NOLOGolden GateING data changes will be silently ignored without any warning.
  •  Golden Gate doesn't support compression on the source database.
  •  Golden Gate does support DDL replication but it is not easy to do selective DDL replication, it replicates every DDL that happens on the source database which is not desirable for some customers.
  • Tables being replicated to on the target can also be written to by any other application or DBA's.
  • Golden Gate supports ignoring data conflicts for updates after the first instantiation of the target database until it catches up. But it is very easy to forget turning off that parameter and any updates being lost will not be alerted by Golden Gate.
  • Golden Gate still works by reverse engineering the Oracle redolog. This may not be totally true with Golden Gate 11, but I expect Golden Gate to interpret Oracle redo more directly in later versions of 11 or 12.
  • Golden Gate dynamically decides to change the key columns that form the supplemental logging based on the state of primary key (i.e. in VALIDATED or NONVALIDATED state), which can introduce data corruptions on the target databases as the expected key columns data is missing in the trail files and they will be set to NULL. They now have the patch available for this, you can set "_USEALLKEYCOLUMNS and ALLOWNONVALIDATEDKEYS" parameters in GLOBALS file to get around this problem.

mongo DB

MongoDB is a document database that provides high performance, high availability, and easy scalability.

Document Database

  • Documents (objects) map nicely to programming language data types.
  • Embedded documents and arrays reduce need for joins.
  • Dynamic schema makes polymorphism easier.
High Performance
  • Embedding makes reads and writes fast.
  • Indexes can include keys from embedded documents and arrays.
  • Optional streaming writes (no acknowledgments).
High Availability
  • Replicated servers with automatic master failover.
Easy Scalability
  • Automatic sharding distributes collection data across machines.
  • Eventually-consistent reads can be distributed over replicated servers.
Advanced Operations

MongoDB Data Model

A MongoDB deployment hosts a number of databases. A manual:database holds a set of collections. Amanual:collection holds a set of documents. A manual:document is a set of key-value pairs. Documents have dynamic schema. Dynamic schema means that documents in the same collection do not need to have the same set of fields or structure, and common fields in a collection’s documents may hold different types of data.

MongoDB Queries

Queries in MongoDB provides a set of operators to define how the find() method selects documents from a collection based on a query specification document that uses a combination of exact equality matches and conditionals using a query operator.

Deployment Architectures

Although MongoDB supports a “standalone” or single-instance operation, production MongoDB deployments are distributed by default. Replica sets provide high performance replication with automated failover, while sharded clusters make it possible to partition large data sets over many machines transparently to the users. MongoDB users combine replica sets and sharded clusters to provide high levels redundancy for large data sets transparently for applications.

MongoDB Design Philosophy


MongoDB wasn’t designed in a lab. We built MongoDB from our own experiences building large scale, high availability, robust systems. We didn’t start from scratch, we really tried to figure out what was broken, and tackle that. So the way I think about MongoDB is that if you take MySql, and change the data model from relational to document based, you get a lot of great features: embedded docs for speed, manageability, agile development with schema-less databases, easier horizontal scalability because joins aren’t as important. There are lots of things that work great in relational databases: indexes, dynamic queries and updates to name a few, and we haven’t changed much there. For example, the way you design your indexes in MongoDB should be exactly the way you do it in MySql or Oracle, you just have the option of indexing an embedded field.

—Eliot Horowitz, MongoDB CTO and Co-founder
  • New database technologies are needed to facilitate horizontal scaling of the data layer, easier development, and the ability to store order(s) of magnitude more data than was used in the past.
  • A non-relational approach is the best path to database solutions which scale horizontally to many machines.
  • It is unacceptable if these new technologies make writing applications harder. Writing code should be faster, easier, and more agile.
  • The document data model (JSON/BSON) is easy to code to, easy to manage(dynamic schema), and yields excellent performance by grouping relevant data together internally.
  • It is important to keep deep functionality to keep programming fast and simple. While some things must be left out, keep as much as possible – for example secondaries indexes, unique key constraints, atomic operations, multi-document updates.
  • Database technology should run anywhere, being available both for running on your own servers or VMs, and also as a cloud pay-for-what-you-use service.

Key MongoDB Features

MongoDB focuses on flexibility, power, speed, and ease of use:

Flexibility

MongoDB stores data in JSON documents (which we serialize to BSON). JSON provides a rich data model that seamlessly maps to native programming language types, and the dynamic schema makes it easier to evolve your data model than with a system with enforced schemas such as a RDBMS.

Power

MongoDB provides a lot of the features of a traditional RDBMS such as secondary indexes, dynamic queries, sorting, rich updates, upserts (update if document exists, insert if it doesn’t), and easy aggregation. This gives you the breadth of functionality that you are used to from an RDBMS, with the flexibility and scaling capability that the non-relational model allows.
Speed/Scaling

By keeping related data together in documents, queries can be much faster than in a relational database where related data is separated into multiple tables and then needs to be joined later. MongoDB also makes it easy to scale out your database. Autosharding allows you to scale your cluster linearly by adding more machines. It is possible to increase capacity without any downtime, which is very important on the web when load can increase suddenly and bringing down the website for extended maintenance can cost your business large amounts of revenue.

Ease of use

MongoDB works hard to be very easy to install, configure, maintain, and use. To this end, MongoDB provides few configuration options, and instead tries to automatically do the “right thing” whenever possible. This means that MongoDB works right out of the box, and you can dive right into developing your application, instead of spending a lot of time fine-tuning obscure database configurations.

Operations


MongoDB is a server process that runs on Linux, Windows and OS X. It can be run both as a 32 or 64-bit application. We recommend running in 64-bit mode, since MongoDB is limited to a total data size of about 2GB for all databases in 32-bit mode.

The MongoDB process listens on port 27017 by default (note that this can be set at start time - please see mongod options for more information).

Clients connect to the MongoDB process, optionally authenticate themselves if security is turned on, and perform a sequence of actions, such as inserts, queries and updates.

MongoDB stores its data in files (default location is /data/db/), and uses memory mapped files for data management for efficiency.

MongoDB can also be configured for data replication.

Additionally the MongoDB Management Service (MMS) application for managing MongoDB clusters using a simple user interface. MMS provides backup and monitoring. MMS is available to all users in the cloud and on-premises as part of MongoDB Standard and Enterprise Subscriptions.Agile and Scalable

MongoDB (from "humongous") is an open-source document database, and the leading NoSQL database. Written in C++, MongoDB features:
Document-Oriented Storage »

JSON-style documents with dynamic schemas offer simplicity and power.
Full Index Support »

Index on any attribute, just like you're used to.
Replication & High Availability »

Mirror across LANs and WANs for scale and peace of mind.
Auto-Sharding »

Scale horizontally without compromising functionality.
Querying »

Rich, document-based queries.
Fast In-Place Updates »

Atomic modifiers for contention-free performance.
Map/Reduce »

Flexible aggregation and data processing.
GridFS »

Store files of any size without complicating your stack.
MongoDB Management Service »

Manage MongoDB on the cloud infrastructure of your choice.
MongoDB Enterprise »

The best way to run MongoDB in production. Secured. Supported. Certified.
Production Support »

Our experts at your fingertips. Get access to our global support organization 24x365