Drizzled Public API Documentation

sync0sync.cc

00001 /*****************************************************************************
00002 
00003 Copyright (C) 1995, 2010, Innobase Oy. All Rights Reserved.
00004 Copyright (C) 2008, Google Inc.
00005 
00006 Portions of this file contain modifications contributed and copyrighted by
00007 Google, Inc. Those modifications are gratefully acknowledged and are described
00008 briefly in the InnoDB documentation. The contributions by Google are
00009 incorporated with their permission, and subject to the conditions contained in
00010 the file COPYING.Google.
00011 
00012 This program is free software; you can redistribute it and/or modify it under
00013 the terms of the GNU General Public License as published by the Free Software
00014 Foundation; version 2 of the License.
00015 
00016 This program is distributed in the hope that it will be useful, but WITHOUT
00017 ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
00018 FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
00019 
00020 You should have received a copy of the GNU General Public License along with
00021 this program; if not, write to the Free Software Foundation, Inc., 51 Franklin
00022 St, Fifth Floor, Boston, MA 02110-1301 USA
00023 
00024 *****************************************************************************/
00025 
00026 /**************************************************/
00033 #include "sync0sync.h"
00034 #ifdef UNIV_NONINL
00035 #include "sync0sync.ic"
00036 #endif
00037 
00038 #include "sync0rw.h"
00039 #include "buf0buf.h"
00040 #include "srv0srv.h"
00041 #include "buf0types.h"
00042 #include "os0sync.h" /* for HAVE_ATOMIC_BUILTINS */
00043 #ifdef UNIV_SYNC_DEBUG
00044 # include "srv0start.h" /* srv_is_being_started */
00045 #endif /* UNIV_SYNC_DEBUG */
00046 
00047 /*
00048   REASONS FOR IMPLEMENTING THE SPIN LOCK MUTEX
00049   ============================================
00050 
00051 Semaphore operations in operating systems are slow: Solaris on a 1993 Sparc
00052 takes 3 microseconds (us) for a lock-unlock pair and Windows NT on a 1995
00053 Pentium takes 20 microseconds for a lock-unlock pair. Therefore, we have to
00054 implement our own efficient spin lock mutex. Future operating systems may
00055 provide efficient spin locks, but we cannot count on that.
00056 
00057 Another reason for implementing a spin lock is that on multiprocessor systems
00058 it can be more efficient for a processor to run a loop waiting for the
00059 semaphore to be released than to switch to a different thread. A thread switch
00060 takes 25 us on both platforms mentioned above. See Gray and Reuter's book
00061 Transaction processing for background.
00062 
00063 How long should the spin loop last before suspending the thread? On a
00064 uniprocessor, spinning does not help at all, because if the thread owning the
00065 mutex is not executing, it cannot be released. Spinning actually wastes
00066 resources.
00067 
00068 On a multiprocessor, we do not know if the thread owning the mutex is
00069 executing or not. Thus it would make sense to spin as long as the operation
00070 guarded by the mutex would typically last assuming that the thread is
00071 executing. If the mutex is not released by that time, we may assume that the
00072 thread owning the mutex is not executing and suspend the waiting thread.
00073 
00074 A typical operation (where no i/o involved) guarded by a mutex or a read-write
00075 lock may last 1 - 20 us on the current Pentium platform. The longest
00076 operations are the binary searches on an index node.
00077 
00078 We conclude that the best choice is to set the spin time at 20 us. Then the
00079 system should work well on a multiprocessor. On a uniprocessor we have to
00080 make sure that thread swithches due to mutex collisions are not frequent,
00081 i.e., they do not happen every 100 us or so, because that wastes too much
00082 resources. If the thread switches are not frequent, the 20 us wasted in spin
00083 loop is not too much.
00084 
00085 Empirical studies on the effect of spin time should be done for different
00086 platforms.
00087 
00088 
00089   IMPLEMENTATION OF THE MUTEX
00090   ===========================
00091 
00092 For background, see Curt Schimmel's book on Unix implementation on modern
00093 architectures. The key points in the implementation are atomicity and
00094 serialization of memory accesses. The test-and-set instruction (XCHG in
00095 Pentium) must be atomic. As new processors may have weak memory models, also
00096 serialization of memory references may be necessary. The successor of Pentium,
00097 P6, has at least one mode where the memory model is weak. As far as we know,
00098 in Pentium all memory accesses are serialized in the program order and we do
00099 not have to worry about the memory model. On other processors there are
00100 special machine instructions called a fence, memory barrier, or storage
00101 barrier (STBAR in Sparc), which can be used to serialize the memory accesses
00102 to happen in program order relative to the fence instruction.
00103 
00104 Leslie Lamport has devised a "bakery algorithm" to implement a mutex without
00105 the atomic test-and-set, but his algorithm should be modified for weak memory
00106 models. We do not use Lamport's algorithm, because we guess it is slower than
00107 the atomic test-and-set.
00108 
00109 Our mutex implementation works as follows: After that we perform the atomic
00110 test-and-set instruction on the memory word. If the test returns zero, we
00111 know we got the lock first. If the test returns not zero, some other thread
00112 was quicker and got the lock: then we spin in a loop reading the memory word,
00113 waiting it to become zero. It is wise to just read the word in the loop, not
00114 perform numerous test-and-set instructions, because they generate memory
00115 traffic between the cache and the main memory. The read loop can just access
00116 the cache, saving bus bandwidth.
00117 
00118 If we cannot acquire the mutex lock in the specified time, we reserve a cell
00119 in the wait array, set the waiters byte in the mutex to 1. To avoid a race
00120 condition, after setting the waiters byte and before suspending the waiting
00121 thread, we still have to check that the mutex is reserved, because it may
00122 have happened that the thread which was holding the mutex has just released
00123 it and did not see the waiters byte set to 1, a case which would lead the
00124 other thread to an infinite wait.
00125 
00126 LEMMA 1: After a thread resets the event of a mutex (or rw_lock), some
00127 =======
00128 thread will eventually call os_event_set() on that particular event.
00129 Thus no infinite wait is possible in this case.
00130 
00131 Proof:  After making the reservation the thread sets the waiters field in the
00132 mutex to 1. Then it checks that the mutex is still reserved by some thread,
00133 or it reserves the mutex for itself. In any case, some thread (which may be
00134 also some earlier thread, not necessarily the one currently holding the mutex)
00135 will set the waiters field to 0 in mutex_exit, and then call
00136 os_event_set() with the mutex as an argument.
00137 Q.E.D.
00138 
00139 LEMMA 2: If an os_event_set() call is made after some thread has called
00140 =======
00141 the os_event_reset() and before it starts wait on that event, the call
00142 will not be lost to the second thread. This is true even if there is an
00143 intervening call to os_event_reset() by another thread.
00144 Thus no infinite wait is possible in this case.
00145 
00146 Proof (non-windows platforms): os_event_reset() returns a monotonically
00147 increasing value of signal_count. This value is increased at every
00148 call of os_event_set() If thread A has called os_event_reset() followed
00149 by thread B calling os_event_set() and then some other thread C calling
00150 os_event_reset(), the is_set flag of the event will be set to FALSE;
00151 but now if thread A calls os_event_wait_low() with the signal_count
00152 value returned from the earlier call of os_event_reset(), it will
00153 return immediately without waiting.
00154 Q.E.D.
00155 
00156 Proof (windows): If there is a writer thread which is forced to wait for
00157 the lock, it may be able to set the state of rw_lock to RW_LOCK_WAIT_EX
00158 The design of rw_lock ensures that there is one and only one thread
00159 that is able to change the state to RW_LOCK_WAIT_EX and this thread is
00160 guaranteed to acquire the lock after it is released by the current
00161 holders and before any other waiter gets the lock.
00162 On windows this thread waits on a separate event i.e.: wait_ex_event.
00163 Since only one thread can wait on this event there is no chance
00164 of this event getting reset before the writer starts wait on it.
00165 Therefore, this thread is guaranteed to catch the os_set_event()
00166 signalled unconditionally at the release of the lock.
00167 Q.E.D. */
00168 
00169 /* Number of spin waits on mutexes: for performance monitoring */
00170 
00173 static ib_int64_t mutex_spin_round_count    = 0;
00176 static ib_int64_t mutex_spin_wait_count   = 0;
00179 static ib_int64_t mutex_os_wait_count   = 0;
00182 UNIV_INTERN ib_int64_t  mutex_exit_count    = 0;
00183 
00186 UNIV_INTERN sync_array_t* sync_primary_wait_array;
00187 
00189 UNIV_INTERN ibool sync_initialized  = FALSE;
00190 
00192 typedef struct sync_level_struct  sync_level_t;
00194 typedef struct sync_thread_struct sync_thread_t;
00195 
00196 #ifdef UNIV_SYNC_DEBUG
00197 
00200 UNIV_INTERN sync_thread_t*  sync_thread_level_arrays;
00201 
00203 UNIV_INTERN mutex_t   sync_thread_mutex;
00204 
00205 # ifdef UNIV_PFS_MUTEX
00206 UNIV_INTERN mysql_pfs_key_t sync_thread_mutex_key;
00207 # endif /* UNIV_PFS_MUTEX */
00208 #endif /* UNIV_SYNC_DEBUG */
00209 
00211 UNIV_INTERN ut_list_base_node_t  mutex_list;
00212 
00214 UNIV_INTERN mutex_t mutex_list_mutex;
00215 
00216 #ifdef UNIV_PFS_MUTEX
00217 UNIV_INTERN mysql_pfs_key_t mutex_list_mutex_key;
00218 #endif /* UNIV_PFS_MUTEX */
00219 
00220 #ifdef UNIV_SYNC_DEBUG
00221 
00222 UNIV_INTERN ibool sync_order_checks_on  = FALSE;
00223 #endif /* UNIV_SYNC_DEBUG */
00224 
00226 struct sync_thread_struct{
00227   os_thread_id_t  id; 
00228   sync_level_t* levels; 
00230 };
00231 
00233 #define SYNC_THREAD_N_LEVELS  10000
00234 
00236 struct sync_level_struct{
00237   void* latch;  
00239   ulint level;  
00240 };
00241 
00242 /******************************************************************/
00247 UNIV_INTERN
00248 void
00249 mutex_create_func(
00250 /*==============*/
00251   mutex_t*  mutex,    
00252 #ifdef UNIV_DEBUG
00253   const char* cmutex_name,  
00254 # ifdef UNIV_SYNC_DEBUG
00255   ulint   level,    
00256 # endif /* UNIV_SYNC_DEBUG */
00257 #endif /* UNIV_DEBUG */
00258   const char* cfile_name, 
00259   ulint   cline)    
00260 {
00261 #if defined(HAVE_ATOMIC_BUILTINS)
00262   mutex_reset_lock_word(mutex);
00263 #else
00264   os_fast_mutex_init(&(mutex->os_fast_mutex));
00265   mutex->lock_word = 0;
00266 #endif
00267   mutex->event = os_event_create(NULL);
00268   mutex_set_waiters(mutex, 0);
00269 #ifdef UNIV_DEBUG
00270   mutex->magic_n = MUTEX_MAGIC_N;
00271 #endif /* UNIV_DEBUG */
00272 #ifdef UNIV_SYNC_DEBUG
00273   mutex->line = 0;
00274   mutex->file_name = "not yet reserved";
00275   mutex->level = level;
00276 #endif /* UNIV_SYNC_DEBUG */
00277   mutex->cfile_name = cfile_name;
00278   mutex->cline = cline;
00279   mutex->count_os_wait = 0;
00280 #ifdef UNIV_DEBUG
00281   mutex->cmutex_name=   cmutex_name;
00282   mutex->count_using=   0;
00283   mutex->mutex_type=    0;
00284   mutex->lspent_time=   0;
00285   mutex->lmax_spent_time=     0;
00286   mutex->count_spin_loop= 0;
00287   mutex->count_spin_rounds=   0;
00288   mutex->count_os_yield=  0;
00289 #endif /* UNIV_DEBUG */
00290 
00291   /* Check that lock_word is aligned; this is important on Intel */
00292   ut_ad(((ulint)(&(mutex->lock_word))) % 4 == 0);
00293 
00294   /* NOTE! The very first mutexes are not put to the mutex list */
00295 
00296   if ((mutex == &mutex_list_mutex)
00297 #ifdef UNIV_SYNC_DEBUG
00298       || (mutex == &sync_thread_mutex)
00299 #endif /* UNIV_SYNC_DEBUG */
00300       ) {
00301 
00302     return;
00303   }
00304 
00305   mutex_enter(&mutex_list_mutex);
00306 
00307   ut_ad(UT_LIST_GET_LEN(mutex_list) == 0
00308         || UT_LIST_GET_FIRST(mutex_list)->magic_n == MUTEX_MAGIC_N);
00309 
00310   UT_LIST_ADD_FIRST(list, mutex_list, mutex);
00311 
00312   mutex_exit(&mutex_list_mutex);
00313 }
00314 
00315 /******************************************************************/
00320 UNIV_INTERN
00321 void
00322 mutex_free_func(
00323 /*============*/
00324   mutex_t*  mutex)  
00325 {
00326   ut_ad(mutex_validate(mutex));
00327   ut_a(mutex_get_lock_word(mutex) == 0);
00328   ut_a(mutex_get_waiters(mutex) == 0);
00329 
00330 #ifdef UNIV_MEM_DEBUG
00331   if (mutex == &mem_hash_mutex) {
00332     ut_ad(UT_LIST_GET_LEN(mutex_list) == 1);
00333     ut_ad(UT_LIST_GET_FIRST(mutex_list) == &mem_hash_mutex);
00334     UT_LIST_REMOVE(list, mutex_list, mutex);
00335     goto func_exit;
00336   }
00337 #endif /* UNIV_MEM_DEBUG */
00338 
00339   if (mutex != &mutex_list_mutex
00340 #ifdef UNIV_SYNC_DEBUG
00341       && mutex != &sync_thread_mutex
00342 #endif /* UNIV_SYNC_DEBUG */
00343       ) {
00344 
00345     mutex_enter(&mutex_list_mutex);
00346 
00347     ut_ad(!UT_LIST_GET_PREV(list, mutex)
00348           || UT_LIST_GET_PREV(list, mutex)->magic_n
00349           == MUTEX_MAGIC_N);
00350     ut_ad(!UT_LIST_GET_NEXT(list, mutex)
00351           || UT_LIST_GET_NEXT(list, mutex)->magic_n
00352           == MUTEX_MAGIC_N);
00353 
00354     UT_LIST_REMOVE(list, mutex_list, mutex);
00355 
00356     mutex_exit(&mutex_list_mutex);
00357   }
00358 
00359   os_event_free(mutex->event);
00360 #ifdef UNIV_MEM_DEBUG
00361 func_exit:
00362 #endif /* UNIV_MEM_DEBUG */
00363 #if !defined(HAVE_ATOMIC_BUILTINS)
00364   os_fast_mutex_free(&(mutex->os_fast_mutex));
00365 #endif
00366   /* If we free the mutex protecting the mutex list (freeing is
00367   not necessary), we have to reset the magic number AFTER removing
00368   it from the list. */
00369 #ifdef UNIV_DEBUG
00370   mutex->magic_n = 0;
00371 #endif /* UNIV_DEBUG */
00372 }
00373 
00374 /********************************************************************/
00379 UNIV_INTERN
00380 ulint
00381 mutex_enter_nowait_func(
00382 /*====================*/
00383   mutex_t*  mutex,    
00384   const char* /*file_name __attribute__((unused))*/,
00387   ulint   /*line __attribute__((unused))*/)
00389 {
00390   ut_ad(mutex_validate(mutex));
00391 
00392   if (!mutex_test_and_set(mutex)) {
00393 
00394     ut_d(mutex->thread_id = os_thread_get_curr_id());
00395 #ifdef UNIV_SYNC_DEBUG
00396     mutex_set_debug_info(mutex, file_name, line);
00397 #endif
00398 
00399     return(0);  /* Succeeded! */
00400   }
00401 
00402   return(1);
00403 }
00404 
00405 #ifdef UNIV_DEBUG
00406 /******************************************************************/
00409 UNIV_INTERN
00410 ibool
00411 mutex_validate(
00412 /*===========*/
00413   const mutex_t*  mutex)  
00414 {
00415   ut_a(mutex);
00416   ut_a(mutex->magic_n == MUTEX_MAGIC_N);
00417 
00418   return(TRUE);
00419 }
00420 
00421 /******************************************************************/
00425 UNIV_INTERN
00426 ibool
00427 mutex_own(
00428 /*======*/
00429   const mutex_t*  mutex)  
00430 {
00431   ut_ad(mutex_validate(mutex));
00432 
00433   return(mutex_get_lock_word(mutex) == 1
00434          && os_thread_eq(mutex->thread_id, os_thread_get_curr_id()));
00435 }
00436 #endif /* UNIV_DEBUG */
00437 
00438 /******************************************************************/
00440 UNIV_INTERN
00441 void
00442 mutex_set_waiters(
00443 /*==============*/
00444   mutex_t*  mutex,  
00445   ulint   n)  
00446 {
00447   volatile ulint* ptr;    /* declared volatile to ensure that
00448           the value is stored to memory */
00449   ut_ad(mutex);
00450 
00451   ptr = &(mutex->waiters);
00452 
00453   *ptr = n;   /* Here we assume that the write of a single
00454         word in memory is atomic */
00455 }
00456 
00457 /******************************************************************/
00461 UNIV_INTERN
00462 void
00463 mutex_spin_wait(
00464 /*============*/
00465   mutex_t*  mutex,    
00466   const char* file_name,  
00468   ulint   line)   
00469 {
00470   ulint    index; /* index of the reserved wait cell */
00471   ulint    i;   /* spin round count */
00472 #ifdef UNIV_DEBUG
00473   ib_int64_t lstart_time = 0, lfinish_time; /* for timing os_wait */
00474   ulint ltime_diff;
00475   ulint sec;
00476   ulint ms;
00477   uint timer_started = 0;
00478 #endif /* UNIV_DEBUG */
00479   ut_ad(mutex);
00480 
00481   /* This update is not thread safe, but we don't mind if the count
00482   isn't exact. Moved out of ifdef that follows because we are willing
00483   to sacrifice the cost of counting this as the data is valuable.
00484   Count the number of calls to mutex_spin_wait. */
00485   mutex_spin_wait_count++;
00486 
00487 mutex_loop:
00488 
00489   i = 0;
00490 
00491   /* Spin waiting for the lock word to become zero. Note that we do
00492   not have to assume that the read access to the lock word is atomic,
00493   as the actual locking is always committed with atomic test-and-set.
00494   In reality, however, all processors probably have an atomic read of
00495   a memory word. */
00496 
00497 spin_loop:
00498   ut_d(mutex->count_spin_loop++);
00499 
00500   while (mutex_get_lock_word(mutex) != 0 && i < SYNC_SPIN_ROUNDS) {
00501     if (srv_spin_wait_delay) {
00502       ut_delay(ut_rnd_interval(0, srv_spin_wait_delay));
00503     }
00504 
00505     i++;
00506   }
00507 
00508   if (i == SYNC_SPIN_ROUNDS) {
00509 #ifdef UNIV_DEBUG
00510     mutex->count_os_yield++;
00511 #ifndef UNIV_HOTBACKUP
00512     if (timed_mutexes && timer_started == 0) {
00513       ut_usectime(&sec, &ms);
00514       lstart_time= (ib_int64_t)sec * 1000000 + ms;
00515       timer_started = 1;
00516     }
00517 #endif /* UNIV_HOTBACKUP */
00518 #endif /* UNIV_DEBUG */
00519     os_thread_yield();
00520   }
00521 
00522 #ifdef UNIV_SRV_PRINT_LATCH_WAITS
00523   fprintf(stderr,
00524     "Thread %lu spin wait mutex at %p"
00525     " cfile %s cline %lu rnds %lu\n",
00526     (ulong) os_thread_pf(os_thread_get_curr_id()), (void*) mutex,
00527     mutex->cfile_name, (ulong) mutex->cline, (ulong) i);
00528 #endif
00529 
00530   mutex_spin_round_count += i;
00531 
00532   ut_d(mutex->count_spin_rounds += i);
00533 
00534   if (mutex_test_and_set(mutex) == 0) {
00535     /* Succeeded! */
00536 
00537     ut_d(mutex->thread_id = os_thread_get_curr_id());
00538 #ifdef UNIV_SYNC_DEBUG
00539     mutex_set_debug_info(mutex, file_name, line);
00540 #endif
00541 
00542     goto finish_timing;
00543   }
00544 
00545   /* We may end up with a situation where lock_word is 0 but the OS
00546   fast mutex is still reserved. On FreeBSD the OS does not seem to
00547   schedule a thread which is constantly calling pthread_mutex_trylock
00548   (in mutex_test_and_set implementation). Then we could end up
00549   spinning here indefinitely. The following 'i++' stops this infinite
00550   spin. */
00551 
00552   i++;
00553 
00554   if (i < SYNC_SPIN_ROUNDS) {
00555     goto spin_loop;
00556   }
00557 
00558   sync_array_reserve_cell(sync_primary_wait_array, mutex,
00559         SYNC_MUTEX, file_name, line, &index);
00560 
00561   /* The memory order of the array reservation and the change in the
00562   waiters field is important: when we suspend a thread, we first
00563   reserve the cell and then set waiters field to 1. When threads are
00564   released in mutex_exit, the waiters field is first set to zero and
00565   then the event is set to the signaled state. */
00566 
00567   mutex_set_waiters(mutex, 1);
00568 
00569   /* Try to reserve still a few times */
00570   for (i = 0; i < 4; i++) {
00571     if (mutex_test_and_set(mutex) == 0) {
00572       /* Succeeded! Free the reserved wait cell */
00573 
00574       sync_array_free_cell(sync_primary_wait_array, index);
00575 
00576       ut_d(mutex->thread_id = os_thread_get_curr_id());
00577 #ifdef UNIV_SYNC_DEBUG
00578       mutex_set_debug_info(mutex, file_name, line);
00579 #endif
00580 
00581 #ifdef UNIV_SRV_PRINT_LATCH_WAITS
00582       fprintf(stderr, "Thread %lu spin wait succeeds at 2:"
00583         " mutex at %p\n",
00584         (ulong) os_thread_pf(os_thread_get_curr_id()),
00585         (void*) mutex);
00586 #endif
00587 
00588       goto finish_timing;
00589 
00590       /* Note that in this case we leave the waiters field
00591       set to 1. We cannot reset it to zero, as we do not
00592       know if there are other waiters. */
00593     }
00594   }
00595 
00596   /* Now we know that there has been some thread holding the mutex
00597   after the change in the wait array and the waiters field was made.
00598   Now there is no risk of infinite wait on the event. */
00599 
00600 #ifdef UNIV_SRV_PRINT_LATCH_WAITS
00601   fprintf(stderr,
00602     "Thread %lu OS wait mutex at %p cfile %s cline %lu rnds %lu\n",
00603     (ulong) os_thread_pf(os_thread_get_curr_id()), (void*) mutex,
00604     mutex->cfile_name, (ulong) mutex->cline, (ulong) i);
00605 #endif
00606 
00607   mutex_os_wait_count++;
00608 
00609   mutex->count_os_wait++;
00610 #ifdef UNIV_DEBUG
00611   /* !!!!! Sometimes os_wait can be called without os_thread_yield */
00612 #ifndef UNIV_HOTBACKUP
00613   if (timed_mutexes == 1 && timer_started == 0) {
00614     ut_usectime(&sec, &ms);
00615     lstart_time= (ib_int64_t)sec * 1000000 + ms;
00616     timer_started = 1;
00617   }
00618 #endif /* UNIV_HOTBACKUP */
00619 #endif /* UNIV_DEBUG */
00620 
00621   sync_array_wait_event(sync_primary_wait_array, index);
00622   goto mutex_loop;
00623 
00624 finish_timing:
00625 #ifdef UNIV_DEBUG
00626   if (timed_mutexes == 1 && timer_started==1) {
00627     ut_usectime(&sec, &ms);
00628     lfinish_time= (ib_int64_t)sec * 1000000 + ms;
00629 
00630     ltime_diff= (ulint) (lfinish_time - lstart_time);
00631     mutex->lspent_time += ltime_diff;
00632 
00633     if (mutex->lmax_spent_time < ltime_diff) {
00634       mutex->lmax_spent_time= ltime_diff;
00635     }
00636   }
00637 #endif /* UNIV_DEBUG */
00638   return;
00639 }
00640 
00641 /******************************************************************/
00643 UNIV_INTERN
00644 void
00645 mutex_signal_object(
00646 /*================*/
00647   mutex_t*  mutex)  
00648 {
00649   mutex_set_waiters(mutex, 0);
00650 
00651   /* The memory order of resetting the waiters field and
00652   signaling the object is important. See LEMMA 1 above. */
00653   os_event_set(mutex->event);
00654   sync_array_object_signalled(sync_primary_wait_array);
00655 }
00656 
00657 #ifdef UNIV_SYNC_DEBUG
00658 /******************************************************************/
00660 UNIV_INTERN
00661 void
00662 mutex_set_debug_info(
00663 /*=================*/
00664   mutex_t*  mutex,    
00665   const char* file_name,  
00666   ulint   line)   
00667 {
00668   ut_ad(mutex);
00669   ut_ad(file_name);
00670 
00671   sync_thread_add_level(mutex, mutex->level);
00672 
00673   mutex->file_name = file_name;
00674   mutex->line  = line;
00675 }
00676 
00677 /******************************************************************/
00679 UNIV_INTERN
00680 void
00681 mutex_get_debug_info(
00682 /*=================*/
00683   mutex_t*  mutex,    
00684   const char**  file_name,  
00685   ulint*    line,   
00686   os_thread_id_t* thread_id)  
00688 {
00689   ut_ad(mutex);
00690 
00691   *file_name = mutex->file_name;
00692   *line    = mutex->line;
00693   *thread_id = mutex->thread_id;
00694 }
00695 
00696 /******************************************************************/
00698 static
00699 void
00700 mutex_list_print_info(
00701 /*==================*/
00702   FILE* file)   
00703 {
00704   mutex_t*  mutex;
00705   const char* file_name;
00706   ulint   line;
00707   os_thread_id_t  thread_id;
00708   ulint   count   = 0;
00709 
00710   fputs("----------\n"
00711         "MUTEX INFO\n"
00712         "----------\n", file);
00713 
00714   mutex_enter(&mutex_list_mutex);
00715 
00716   mutex = UT_LIST_GET_FIRST(mutex_list);
00717 
00718   while (mutex != NULL) {
00719     count++;
00720 
00721     if (mutex_get_lock_word(mutex) != 0) {
00722       mutex_get_debug_info(mutex, &file_name, &line,
00723                &thread_id);
00724       fprintf(file,
00725         "Locked mutex: addr %p thread %ld"
00726         " file %s line %ld\n",
00727         (void*) mutex, os_thread_pf(thread_id),
00728         file_name, line);
00729     }
00730 
00731     mutex = UT_LIST_GET_NEXT(list, mutex);
00732   }
00733 
00734   fprintf(file, "Total number of mutexes %ld\n", count);
00735 
00736   mutex_exit(&mutex_list_mutex);
00737 }
00738 
00739 /******************************************************************/
00742 UNIV_INTERN
00743 ulint
00744 mutex_n_reserved(void)
00745 /*==================*/
00746 {
00747   mutex_t*  mutex;
00748   ulint   count   = 0;
00749 
00750   mutex_enter(&mutex_list_mutex);
00751 
00752   mutex = UT_LIST_GET_FIRST(mutex_list);
00753 
00754   while (mutex != NULL) {
00755     if (mutex_get_lock_word(mutex) != 0) {
00756 
00757       count++;
00758     }
00759 
00760     mutex = UT_LIST_GET_NEXT(list, mutex);
00761   }
00762 
00763   mutex_exit(&mutex_list_mutex);
00764 
00765   ut_a(count >= 1);
00766 
00767   return(count - 1); /* Subtract one, because this function itself
00768          was holding one mutex (mutex_list_mutex) */
00769 }
00770 
00771 /******************************************************************/
00775 UNIV_INTERN
00776 ibool
00777 sync_all_freed(void)
00778 /*================*/
00779 {
00780   return(mutex_n_reserved() + rw_lock_n_locked() == 0);
00781 }
00782 
00783 /******************************************************************/
00786 static
00787 sync_thread_t*
00788 sync_thread_level_arrays_get_nth(
00789 /*=============================*/
00790   ulint n)  
00791 {
00792   ut_ad(n < OS_THREAD_MAX_N);
00793 
00794   return(sync_thread_level_arrays + n);
00795 }
00796 
00797 /******************************************************************/
00800 static
00801 sync_thread_t*
00802 sync_thread_level_arrays_find_slot(void)
00803 /*====================================*/
00804 
00805 {
00806   sync_thread_t*  slot;
00807   os_thread_id_t  id;
00808   ulint   i;
00809 
00810   id = os_thread_get_curr_id();
00811 
00812   for (i = 0; i < OS_THREAD_MAX_N; i++) {
00813 
00814     slot = sync_thread_level_arrays_get_nth(i);
00815 
00816     if (slot->levels && os_thread_eq(slot->id, id)) {
00817 
00818       return(slot);
00819     }
00820   }
00821 
00822   return(NULL);
00823 }
00824 
00825 /******************************************************************/
00828 static
00829 sync_thread_t*
00830 sync_thread_level_arrays_find_free(void)
00831 /*====================================*/
00832 
00833 {
00834   sync_thread_t*  slot;
00835   ulint   i;
00836 
00837   for (i = 0; i < OS_THREAD_MAX_N; i++) {
00838 
00839     slot = sync_thread_level_arrays_get_nth(i);
00840 
00841     if (slot->levels == NULL) {
00842 
00843       return(slot);
00844     }
00845   }
00846 
00847   return(NULL);
00848 }
00849 
00850 /******************************************************************/
00853 static
00854 sync_level_t*
00855 sync_thread_levels_get_nth(
00856 /*=======================*/
00857   sync_level_t* arr,  
00859   ulint   n)  
00860 {
00861   ut_ad(n < SYNC_THREAD_N_LEVELS);
00862 
00863   return(arr + n);
00864 }
00865 
00866 /******************************************************************/
00870 static
00871 ibool
00872 sync_thread_levels_g(
00873 /*=================*/
00874   sync_level_t* arr,  
00876   ulint   limit,  
00877   ulint   warn) 
00878 {
00879   sync_level_t* slot;
00880   rw_lock_t*  lock;
00881   mutex_t*  mutex;
00882   ulint   i;
00883 
00884   for (i = 0; i < SYNC_THREAD_N_LEVELS; i++) {
00885 
00886     slot = sync_thread_levels_get_nth(arr, i);
00887 
00888     if (slot->latch != NULL) {
00889       if (slot->level <= limit) {
00890 
00891         if (!warn) {
00892 
00893           return(FALSE);
00894         }
00895 
00896         lock = slot->latch;
00897         mutex = slot->latch;
00898 
00899         fprintf(stderr,
00900           "InnoDB: sync levels should be"
00901           " > %lu but a level is %lu\n",
00902           (ulong) limit, (ulong) slot->level);
00903 
00904         if (mutex->magic_n == MUTEX_MAGIC_N) {
00905           fprintf(stderr,
00906             "Mutex created at %s %lu\n",
00907             mutex->cfile_name,
00908             (ulong) mutex->cline);
00909 
00910           if (mutex_get_lock_word(mutex) != 0) {
00911             const char* file_name;
00912             ulint   line;
00913             os_thread_id_t  thread_id;
00914 
00915             mutex_get_debug_info(
00916               mutex, &file_name,
00917               &line, &thread_id);
00918 
00919             fprintf(stderr,
00920               "InnoDB: Locked mutex:"
00921               " addr %p thread %ld"
00922               " file %s line %ld\n",
00923               (void*) mutex,
00924               os_thread_pf(
00925                 thread_id),
00926               file_name,
00927               (ulong) line);
00928           } else {
00929             fputs("Not locked\n", stderr);
00930           }
00931         } else {
00932           rw_lock_print(lock);
00933         }
00934 
00935         return(FALSE);
00936       }
00937     }
00938   }
00939 
00940   return(TRUE);
00941 }
00942 
00943 /******************************************************************/
00946 static
00947 ibool
00948 sync_thread_levels_contain(
00949 /*=======================*/
00950   sync_level_t* arr,  
00952   ulint   level)  
00953 {
00954   sync_level_t* slot;
00955   ulint   i;
00956 
00957   for (i = 0; i < SYNC_THREAD_N_LEVELS; i++) {
00958 
00959     slot = sync_thread_levels_get_nth(arr, i);
00960 
00961     if (slot->latch != NULL) {
00962       if (slot->level == level) {
00963 
00964         return(TRUE);
00965       }
00966     }
00967   }
00968 
00969   return(FALSE);
00970 }
00971 
00972 /******************************************************************/
00976 UNIV_INTERN
00977 void*
00978 sync_thread_levels_contains(
00979 /*========================*/
00980   ulint level)      
00982 {
00983   sync_level_t* arr;
00984   sync_thread_t*  thread_slot;
00985   sync_level_t* slot;
00986   ulint   i;
00987 
00988   if (!sync_order_checks_on) {
00989 
00990     return(NULL);
00991   }
00992 
00993   mutex_enter(&sync_thread_mutex);
00994 
00995   thread_slot = sync_thread_level_arrays_find_slot();
00996 
00997   if (thread_slot == NULL) {
00998 
00999     mutex_exit(&sync_thread_mutex);
01000 
01001     return(NULL);
01002   }
01003 
01004   arr = thread_slot->levels;
01005 
01006   for (i = 0; i < SYNC_THREAD_N_LEVELS; i++) {
01007 
01008     slot = sync_thread_levels_get_nth(arr, i);
01009 
01010     if (slot->latch != NULL && slot->level == level) {
01011 
01012       mutex_exit(&sync_thread_mutex);
01013       return(slot->latch);
01014     }
01015   }
01016 
01017   mutex_exit(&sync_thread_mutex);
01018 
01019   return(NULL);
01020 }
01021 
01022 /******************************************************************/
01025 UNIV_INTERN
01026 void*
01027 sync_thread_levels_nonempty_gen(
01028 /*============================*/
01029   ibool dict_mutex_allowed) 
01033 {
01034   sync_level_t* arr;
01035   sync_thread_t*  thread_slot;
01036   sync_level_t* slot;
01037   ulint   i;
01038 
01039   if (!sync_order_checks_on) {
01040 
01041     return(NULL);
01042   }
01043 
01044   mutex_enter(&sync_thread_mutex);
01045 
01046   thread_slot = sync_thread_level_arrays_find_slot();
01047 
01048   if (thread_slot == NULL) {
01049 
01050     mutex_exit(&sync_thread_mutex);
01051 
01052     return(NULL);
01053   }
01054 
01055   arr = thread_slot->levels;
01056 
01057   for (i = 0; i < SYNC_THREAD_N_LEVELS; i++) {
01058 
01059     slot = sync_thread_levels_get_nth(arr, i);
01060 
01061     if (slot->latch != NULL
01062         && (!dict_mutex_allowed
01063       || (slot->level != SYNC_DICT
01064           && slot->level != SYNC_DICT_OPERATION))) {
01065 
01066       mutex_exit(&sync_thread_mutex);
01067       ut_error;
01068 
01069       return(slot->latch);
01070     }
01071   }
01072 
01073   mutex_exit(&sync_thread_mutex);
01074 
01075   return(NULL);
01076 }
01077 
01078 /******************************************************************/
01081 UNIV_INTERN
01082 ibool
01083 sync_thread_levels_empty(void)
01084 /*==========================*/
01085 {
01086   return(sync_thread_levels_empty_gen(FALSE));
01087 }
01088 
01089 /******************************************************************/
01093 UNIV_INTERN
01094 void
01095 sync_thread_add_level(
01096 /*==================*/
01097   void* latch,  
01098   ulint level)  
01100 {
01101   sync_level_t* array;
01102   sync_level_t* slot;
01103   sync_thread_t*  thread_slot;
01104   ulint   i;
01105 
01106   if (!sync_order_checks_on) {
01107 
01108     return;
01109   }
01110 
01111   if ((latch == (void*)&sync_thread_mutex)
01112       || (latch == (void*)&mutex_list_mutex)
01113       || (latch == (void*)&rw_lock_debug_mutex)
01114       || (latch == (void*)&rw_lock_list_mutex)) {
01115 
01116     return;
01117   }
01118 
01119   if (level == SYNC_LEVEL_VARYING) {
01120 
01121     return;
01122   }
01123 
01124   mutex_enter(&sync_thread_mutex);
01125 
01126   thread_slot = sync_thread_level_arrays_find_slot();
01127 
01128   if (thread_slot == NULL) {
01129     /* We have to allocate the level array for a new thread */
01130     array = ut_malloc(sizeof(sync_level_t) * SYNC_THREAD_N_LEVELS);
01131 
01132     thread_slot = sync_thread_level_arrays_find_free();
01133 
01134     thread_slot->id = os_thread_get_curr_id();
01135     thread_slot->levels = array;
01136 
01137     for (i = 0; i < SYNC_THREAD_N_LEVELS; i++) {
01138 
01139       slot = sync_thread_levels_get_nth(array, i);
01140 
01141       slot->latch = NULL;
01142     }
01143   }
01144 
01145   array = thread_slot->levels;
01146 
01147   /* NOTE that there is a problem with _NODE and _LEAF levels: if the
01148   B-tree height changes, then a leaf can change to an internal node
01149   or the other way around. We do not know at present if this can cause
01150   unnecessary assertion failures below. */
01151 
01152   switch (level) {
01153   case SYNC_NO_ORDER_CHECK:
01154   case SYNC_EXTERN_STORAGE:
01155   case SYNC_TREE_NODE_FROM_HASH:
01156     /* Do no order checking */
01157     break;
01158   case SYNC_TRX_SYS_HEADER:
01159     if (srv_is_being_started) {
01160       /* This is violated during trx_sys_create_rsegs()
01161       when creating additional rollback segments when
01162       upgrading in innobase_start_or_create_for_mysql(). */
01163       break;
01164     }
01165   case SYNC_MEM_POOL:
01166   case SYNC_MEM_HASH:
01167   case SYNC_RECV:
01168   case SYNC_WORK_QUEUE:
01169   case SYNC_LOG:
01170   case SYNC_LOG_FLUSH_ORDER:
01171   case SYNC_THR_LOCAL:
01172   case SYNC_ANY_LATCH:
01173   case SYNC_FILE_FORMAT_TAG:
01174   case SYNC_DOUBLEWRITE:
01175   case SYNC_SEARCH_SYS:
01176   case SYNC_SEARCH_SYS_CONF:
01177   case SYNC_TRX_LOCK_HEAP:
01178   case SYNC_KERNEL:
01179   case SYNC_IBUF_BITMAP_MUTEX:
01180   case SYNC_RSEG:
01181   case SYNC_TRX_UNDO:
01182   case SYNC_PURGE_LATCH:
01183   case SYNC_PURGE_SYS:
01184   case SYNC_DICT_AUTOINC_MUTEX:
01185   case SYNC_DICT_OPERATION:
01186   case SYNC_DICT_HEADER:
01187   case SYNC_TRX_I_S_RWLOCK:
01188   case SYNC_TRX_I_S_LAST_READ:
01189     if (!sync_thread_levels_g(array, level, TRUE)) {
01190       fprintf(stderr,
01191         "InnoDB: sync_thread_levels_g(array, %lu)"
01192         " does not hold!\n", level);
01193       ut_error;
01194     }
01195     break;
01196   case SYNC_BUF_FLUSH_LIST:
01197   case SYNC_BUF_POOL:
01198     /* We can have multiple mutexes of this type therefore we
01199     can only check whether the greater than condition holds. */
01200     if (!sync_thread_levels_g(array, level-1, TRUE)) {
01201       fprintf(stderr,
01202         "InnoDB: sync_thread_levels_g(array, %lu)"
01203         " does not hold!\n", level-1);
01204       ut_error;
01205     }
01206     break;
01207 
01208   case SYNC_BUF_BLOCK:
01209     /* Either the thread must own the buffer pool mutex
01210     (buf_pool->mutex), or it is allowed to latch only ONE
01211     buffer block (block->mutex or buf_pool->zip_mutex). */
01212     if (!sync_thread_levels_g(array, level, FALSE)) {
01213       ut_a(sync_thread_levels_g(array, level - 1, TRUE));
01214       ut_a(sync_thread_levels_contain(array, SYNC_BUF_POOL));
01215     }
01216     break;
01217   case SYNC_REC_LOCK:
01218     if (sync_thread_levels_contain(array, SYNC_KERNEL)) {
01219       ut_a(sync_thread_levels_g(array, SYNC_REC_LOCK - 1,
01220               TRUE));
01221     } else {
01222       ut_a(sync_thread_levels_g(array, SYNC_REC_LOCK, TRUE));
01223     }
01224     break;
01225   case SYNC_IBUF_BITMAP:
01226     /* Either the thread must own the master mutex to all
01227     the bitmap pages, or it is allowed to latch only ONE
01228     bitmap page. */
01229     if (sync_thread_levels_contain(array,
01230                  SYNC_IBUF_BITMAP_MUTEX)) {
01231       ut_a(sync_thread_levels_g(array, SYNC_IBUF_BITMAP - 1,
01232               TRUE));
01233     } else {
01234       /* This is violated during trx_sys_create_rsegs()
01235       when creating additional rollback segments when
01236       upgrading in innobase_start_or_create_for_mysql(). */
01237       ut_a(srv_is_being_started
01238            || sync_thread_levels_g(array, SYNC_IBUF_BITMAP,
01239                  TRUE));
01240     }
01241     break;
01242   case SYNC_FSP_PAGE:
01243     ut_a(sync_thread_levels_contain(array, SYNC_FSP));
01244     break;
01245   case SYNC_FSP:
01246     ut_a(sync_thread_levels_contain(array, SYNC_FSP)
01247          || sync_thread_levels_g(array, SYNC_FSP, TRUE));
01248     break;
01249   case SYNC_TRX_UNDO_PAGE:
01250     ut_a(sync_thread_levels_contain(array, SYNC_TRX_UNDO)
01251          || sync_thread_levels_contain(array, SYNC_RSEG)
01252          || sync_thread_levels_contain(array, SYNC_PURGE_SYS)
01253          || sync_thread_levels_g(array, SYNC_TRX_UNDO_PAGE, TRUE));
01254     break;
01255   case SYNC_RSEG_HEADER:
01256     ut_a(sync_thread_levels_contain(array, SYNC_RSEG));
01257     break;
01258   case SYNC_RSEG_HEADER_NEW:
01259     ut_a(sync_thread_levels_contain(array, SYNC_KERNEL)
01260          && sync_thread_levels_contain(array, SYNC_FSP_PAGE));
01261     break;
01262   case SYNC_TREE_NODE:
01263     ut_a(sync_thread_levels_contain(array, SYNC_INDEX_TREE)
01264          || sync_thread_levels_contain(array, SYNC_DICT_OPERATION)
01265          || sync_thread_levels_g(array, SYNC_TREE_NODE - 1, TRUE));
01266     break;
01267   case SYNC_TREE_NODE_NEW:
01268     ut_a(sync_thread_levels_contain(array, SYNC_FSP_PAGE)
01269          || sync_thread_levels_contain(array, SYNC_IBUF_MUTEX));
01270     break;
01271   case SYNC_INDEX_TREE:
01272     if (sync_thread_levels_contain(array, SYNC_IBUF_MUTEX)
01273         && sync_thread_levels_contain(array, SYNC_FSP)) {
01274       ut_a(sync_thread_levels_g(array, SYNC_FSP_PAGE - 1,
01275               TRUE));
01276     } else {
01277       ut_a(sync_thread_levels_g(array, SYNC_TREE_NODE - 1,
01278               TRUE));
01279     }
01280     break;
01281   case SYNC_IBUF_MUTEX:
01282     ut_a(sync_thread_levels_g(array, SYNC_FSP_PAGE - 1, TRUE));
01283     break;
01284   case SYNC_IBUF_PESS_INSERT_MUTEX:
01285     ut_a(sync_thread_levels_g(array, SYNC_FSP - 1, TRUE));
01286     ut_a(!sync_thread_levels_contain(array, SYNC_IBUF_MUTEX));
01287     break;
01288   case SYNC_IBUF_HEADER:
01289     ut_a(sync_thread_levels_g(array, SYNC_FSP - 1, TRUE));
01290     ut_a(!sync_thread_levels_contain(array, SYNC_IBUF_MUTEX));
01291     ut_a(!sync_thread_levels_contain(array,
01292              SYNC_IBUF_PESS_INSERT_MUTEX));
01293     break;
01294   case SYNC_DICT:
01295 #ifdef UNIV_DEBUG
01296     ut_a(buf_debug_prints
01297          || sync_thread_levels_g(array, SYNC_DICT, TRUE));
01298 #else /* UNIV_DEBUG */
01299     ut_a(sync_thread_levels_g(array, SYNC_DICT, TRUE));
01300 #endif /* UNIV_DEBUG */
01301     break;
01302   default:
01303     ut_error;
01304   }
01305 
01306   for (i = 0; i < SYNC_THREAD_N_LEVELS; i++) {
01307 
01308     slot = sync_thread_levels_get_nth(array, i);
01309 
01310     if (slot->latch == NULL) {
01311       slot->latch = latch;
01312       slot->level = level;
01313 
01314       break;
01315     }
01316   }
01317 
01318   ut_a(i < SYNC_THREAD_N_LEVELS);
01319 
01320   mutex_exit(&sync_thread_mutex);
01321 }
01322 
01323 /******************************************************************/
01328 UNIV_INTERN
01329 ibool
01330 sync_thread_reset_level(
01331 /*====================*/
01332   void* latch)  
01333 {
01334   sync_level_t* array;
01335   sync_level_t* slot;
01336   sync_thread_t*  thread_slot;
01337   ulint   i;
01338 
01339   if (!sync_order_checks_on) {
01340 
01341     return(FALSE);
01342   }
01343 
01344   if ((latch == (void*)&sync_thread_mutex)
01345       || (latch == (void*)&mutex_list_mutex)
01346       || (latch == (void*)&rw_lock_debug_mutex)
01347       || (latch == (void*)&rw_lock_list_mutex)) {
01348 
01349     return(FALSE);
01350   }
01351 
01352   mutex_enter(&sync_thread_mutex);
01353 
01354   thread_slot = sync_thread_level_arrays_find_slot();
01355 
01356   if (thread_slot == NULL) {
01357 
01358     ut_error;
01359 
01360     mutex_exit(&sync_thread_mutex);
01361     return(FALSE);
01362   }
01363 
01364   array = thread_slot->levels;
01365 
01366   for (i = 0; i < SYNC_THREAD_N_LEVELS; i++) {
01367 
01368     slot = sync_thread_levels_get_nth(array, i);
01369 
01370     if (slot->latch == latch) {
01371       slot->latch = NULL;
01372 
01373       mutex_exit(&sync_thread_mutex);
01374 
01375       return(TRUE);
01376     }
01377   }
01378 
01379   if (((mutex_t*) latch)->magic_n != MUTEX_MAGIC_N) {
01380     rw_lock_t*  rw_lock;
01381 
01382     rw_lock = (rw_lock_t*) latch;
01383 
01384     if (rw_lock->level == SYNC_LEVEL_VARYING) {
01385       mutex_exit(&sync_thread_mutex);
01386 
01387       return(TRUE);
01388     }
01389   }
01390 
01391   ut_error;
01392 
01393   mutex_exit(&sync_thread_mutex);
01394 
01395   return(FALSE);
01396 }
01397 #endif /* UNIV_SYNC_DEBUG */
01398 
01399 /******************************************************************/
01401 UNIV_INTERN
01402 void
01403 sync_init(void)
01404 /*===========*/
01405 {
01406 #ifdef UNIV_SYNC_DEBUG
01407   sync_thread_t*  thread_slot;
01408   ulint   i;
01409 #endif /* UNIV_SYNC_DEBUG */
01410 
01411   ut_a(sync_initialized == FALSE);
01412 
01413   sync_initialized = TRUE;
01414 
01415   /* Create the primary system wait array which is protected by an OS
01416   mutex */
01417 
01418   sync_primary_wait_array = sync_array_create(OS_THREAD_MAX_N,
01419                 SYNC_ARRAY_OS_MUTEX);
01420 #ifdef UNIV_SYNC_DEBUG
01421   /* Create the thread latch level array where the latch levels
01422   are stored for each OS thread */
01423 
01424   sync_thread_level_arrays = ut_malloc(OS_THREAD_MAX_N
01425                * sizeof(sync_thread_t));
01426   for (i = 0; i < OS_THREAD_MAX_N; i++) {
01427 
01428     thread_slot = sync_thread_level_arrays_get_nth(i);
01429     thread_slot->levels = NULL;
01430   }
01431 #endif /* UNIV_SYNC_DEBUG */
01432   /* Init the mutex list and create the mutex to protect it. */
01433 
01434   UT_LIST_INIT(mutex_list);
01435   mutex_create(mutex_list_mutex_key, &mutex_list_mutex,
01436          SYNC_NO_ORDER_CHECK);
01437 #ifdef UNIV_SYNC_DEBUG
01438   mutex_create(sync_thread_mutex_key, &sync_thread_mutex,
01439          SYNC_NO_ORDER_CHECK);
01440 #endif /* UNIV_SYNC_DEBUG */
01441 
01442   /* Init the rw-lock list and create the mutex to protect it. */
01443 
01444   UT_LIST_INIT(rw_lock_list);
01445   mutex_create(rw_lock_list_mutex_key, &rw_lock_list_mutex,
01446          SYNC_NO_ORDER_CHECK);
01447 
01448 #ifdef UNIV_SYNC_DEBUG
01449   mutex_create(rw_lock_debug_mutex_key, &rw_lock_debug_mutex,
01450          SYNC_NO_ORDER_CHECK);
01451 
01452   rw_lock_debug_event = os_event_create(NULL);
01453   rw_lock_debug_waiters = FALSE;
01454 #endif /* UNIV_SYNC_DEBUG */
01455 }
01456 
01457 /******************************************************************/
01460 UNIV_INTERN
01461 void
01462 sync_close(void)
01463 /*===========*/
01464 {
01465   mutex_t*  mutex;
01466 
01467   sync_array_free(sync_primary_wait_array);
01468 
01469   mutex = UT_LIST_GET_FIRST(mutex_list);
01470 
01471   while (mutex) {
01472 #ifdef UNIV_MEM_DEBUG
01473     if (mutex == &mem_hash_mutex) {
01474       mutex = UT_LIST_GET_NEXT(list, mutex);
01475       continue;
01476     }
01477 #endif /* UNIV_MEM_DEBUG */
01478     mutex_free(mutex);
01479     mutex = UT_LIST_GET_FIRST(mutex_list);
01480   }
01481 
01482   mutex_free(&mutex_list_mutex);
01483 #ifdef UNIV_SYNC_DEBUG
01484   mutex_free(&sync_thread_mutex);
01485 
01486   /* Switch latching order checks on in sync0sync.c */
01487   sync_order_checks_on = FALSE;
01488 #endif /* UNIV_SYNC_DEBUG */
01489 
01490   sync_initialized = FALSE; 
01491 }
01492 
01493 /*******************************************************************/
01495 UNIV_INTERN
01496 void
01497 sync_print_wait_info(
01498 /*=================*/
01499   FILE* file)   
01500 {
01501 #ifdef UNIV_SYNC_DEBUG
01502   fprintf(file, "Mutex exits %llu, rws exits %llu, rwx exits %llu\n",
01503     mutex_exit_count, rw_s_exit_count, rw_x_exit_count);
01504 #endif
01505 
01506   fprintf(file,
01507     "Mutex spin waits %"PRId64", rounds %"PRId64", "
01508                 "OS waits %"PRId64"\n"
01509     "RW-shared spins %"PRId64", rounds %"PRId64", OS waits %"PRId64";"
01510     " RW-excl spins %"PRId64", rounds %"PRId64", OS waits %"PRId64"\n",
01511     mutex_spin_wait_count,
01512     mutex_spin_round_count,
01513     mutex_os_wait_count,
01514     rw_s_spin_wait_count,
01515                 rw_s_spin_round_count,
01516     rw_s_os_wait_count,
01517     rw_x_spin_wait_count,
01518                 rw_x_spin_round_count,
01519     rw_x_os_wait_count);
01520 
01521   fprintf(file,
01522     "Spin rounds per wait: %.2f mutex, %.2f RW-shared, "
01523     "%.2f RW-excl\n",
01524     (double) mutex_spin_round_count /
01525     (mutex_spin_wait_count ? mutex_spin_wait_count : 1),
01526     (double) rw_s_spin_round_count /
01527     (rw_s_spin_wait_count ? rw_s_spin_wait_count : 1),
01528     (double) rw_x_spin_round_count /
01529     (rw_x_spin_wait_count ? rw_x_spin_wait_count : 1));
01530 }
01531 
01532 /*******************************************************************/
01534 UNIV_INTERN
01535 void
01536 sync_print(
01537 /*=======*/
01538   FILE* file)   
01539 {
01540 #ifdef UNIV_SYNC_DEBUG
01541   mutex_list_print_info(file);
01542 
01543   rw_lock_list_print_info(file);
01544 #endif /* UNIV_SYNC_DEBUG */
01545 
01546   sync_array_print_info(file, sync_primary_wait_array);
01547 
01548   sync_print_wait_info(file);
01549 }