Quantcast
Channel: SCN : Document List - SAP IQ
Viewing all 123 articles
Browse latest View live

Steps to recover RLV from failed recovery

$
0
0

If RLV failed to recover due to any kind of corruption such as OS exceptions/checksum errors on reading page from disk or  mismatched sequence number on head/tail of page.

 

Recovery Steps:

 

  1. Add -gm 1 (allow single user connection plus one DBA connection) and -gd DBA (users with SYSTEM OPERATOR system privilege can start and stop database) in param.cfg
  2. Add -iqrlvrec_bypass 1 switch to params.cfg to by pass all RLV recovery. This option should only be used in emergency to drop corrupted RLV table or for deleting RLV portion of the table.  Currently this option disables further logging, but doesn't prevent general RLV operations. Non-DBA users should n't allowed access to database, when recovering RLV with this switch.
  3. Run sp_iqcheckdb 'allocation/check/verify database' to check consistency of the database.
  4. Delete RLV portion of the table(you will lose all the data in the RLV portion of the table), This might leave inconsistent database, but will allow a subsequent recovery

              delete * from table where rowid() > 2^48

 

    5. Stop and restart SAP Sybase IQ server normally i.e without  -gm 1 -gd DBA and -iqrlvrec_bypass 1 switches.


Resolving RLV Store Out of Memory Error

$
0
0

Issue: RLV Store has run out of memory. SQLCODE= -1013129, ODBC 3 State="HY000"

 

This error can be raised due to any one of the following causes.

 

1. Too much data being loaded/updated  by RLV transactions

 

Solution: Reduce size of loads/updates. Keep size of updates per transaction small by issuing frequent commits.

 

2. Automated merge is not able to keep up with RLV workload.

 

Solution:

 

  •     Review SYQIQRLVMERGEHISTORY, merge history system view. A log entry for each RLV enabled-table is added each time a mege between RLV Store and IQ Main Store is started and is updated when a merge completes.

 

  •     Adjust database option RV_AUTO_MERGE_EVAL_INTERVAL, for configuring the evalution period in minutes to determine an automated merge of RLV and IQ Main Stores should occur.  Default : 15 minutes

 

  •   Set database options RV_MERGE_TABLE_NUMROWS, RV_MERGE_TABLE_MEMPERCENT, and RV_MERGE_NODE_MEMSIZE for adjusting automated merge thresholds

 

3. Long running transaction(s) might be holding old RLV store fragments in memory.

 

Solution:

 

  • Run sp_iqrlvmemory System Stored Procedure to determine whether old RLV store fragments exist.

 

  • Execute sp_iqtransaction System Stored Procedure to locate old active transaction, then use drop connection <conn Id> to terminate connection.

 

4.  Maximum RLV Memory configured values is too low.

 

Solution:

 

  • increase max RLV memory, using IQ server startup switch -iqrlvmem.

 

In some scenarios, if  you are loading lot of data and load  and manual merge both are  failing with 'out of RLV memory error' and you  can not  restart IQ server to increase max RLV memory. In that scenario:

 

  • If you physical RAM available on machine, Increase RLV memory run time using sa_server_option.

          

              sa_server_option 'rlv_memory_mb', <maximum memory in mb>

 

  • Manual blocking merge to free up RLV memory.

       

        sp_iqmergerlvstore (merge_type, table_name, [table_owner])

 

          - Merge_type can be Blocking | Non_Blocking.  For manual merge use Blocking merge, because  Non_blocking merge need additional in-memory table fragments, which wiil not be possible, if RLV memory is already low/out of memory.

Hash Insert Buffer Unpinning Messages in SAP Sybase IQ 15

$
0
0

Synopsis

This article describes what causes the message “Warning: Hash Insert forced buffer unpinning detected for FP Index” to be logged in SAP Sybase IQ 15 and offers recommendations for what to do about it.

 

SAP Sybase IQ Definitions

 

IQ Temporary Cache – A fixed amount of RAM on the system running SAP Sybase IQ is allocated as working memory for transitory tasks including table loading, sorting, and aggregation operations.

 

FP index – The Fast Projection index is the default index storage for column values in a permanent table.

 

Optimized FP index – Small numbers of distinct or unique column values allow an FP index to be optimized in the form of a lookup table where each distinct value is stored once and an array of column references pointing to one key in the lookup table. FP(1) indexes use one byte to index up to 256 distinct lookup values. FP(2) indexes allow up to 65536 distinct values. FP(3) indexes, new in IQ 15, can have up to 16777216 lookup values. FP(3) indexes can grow very large.

 

FP_LOOKUP_SIZE, FP_LOOKUP_SIZE_PPM – These options provide a limit on how many IQ cache pages can be used for a lookup table for an optimized FP index. They can be an important control for FP(3) indexes.

 

.iqmsg log – The SAP Sybase IQ database maintains a log file with “.iqmsg” extension. It shows startup information, IQ transaction manager activity, and database-level warning and error messages.

 

Warning messages for “buffer unpinning detected” in the .iqmsg log

 

The IQ 15 .iqmsg log may show warning messages when loading or inserting to a table as in the following example:

 

I. 05/02 12:35:16. 0000251232 Txn 885667144 0 885667144

I. 05/02 12:35:16. 0000251232 Insert Started.

I. 05/02 12:35:16. 0000251232 AdjustmentMade

I. 05/02 12:35:17. 0000251232 Warning: Hash Insert forced buffer unpinning detected for FP Index GSinfra.AdjustmentMade.ASIQ_IDX_T5274_C19_FP.

I. 05/02 12:35:17. 0000251232 [20895]: Insert Pass 1 completed in 1 seconds.

I. 05/02 12:35:17. 0000251232 [20895]: Insert Pass 2 completed in 0 seconds.

I. 05/02 12:35:17. 0000251232 [20834]:

        1862 records were inserted into 'AdjustmentMade'.

I. 05/02 12:35:17. 0000251232 [20896]: Insert for 'AdjustmentMade' completed in 1 seconds.  1862 rows inserted.

I. 05/02 12:35:18. 0000251232 Cmt 885667147

I. 05/02 12:35:18. 0000251232 PostCmt 0

 

The purpose of the message is explained in the following excerpt from the topic “Configuring FP(3) Indexes” in the IQ 15 System Administration Guide:

 

FP(3) Loads

Columns with a 3-byte index require additional cache to load data. Set FP_LOOKUP_SIZE to an appropriate value before loading columns with 3-bytes indexes.

If a scarcity of pinned buffers occurs, Sybase IQ returns a warning in the .iqmsg file, which also contains notification of possible thrashing:

Warning: Hash Insert forced buffer unpinning detected for FP Index

Warning: Hash Insert thrashing detected for FP Index

 

Impact of the warning

 

Occurrence of this message indicates that sufficient memory was not readily available during the process of building an FP(3) index in IQ Temporary Cache. The server did acquire a memory page buffer, by finding one that was unused, but it had to look for it. This means the allocation of IQ Temporary Cache buffers available to the user receiving the message had run low. As long as the server can find unused buffers when needed, there is essentially no impact on performance. However, if no unused cache buffers can be found, then buffers in use may be written out to IQ Store on disk to free them for reuse. When needed later, they would be read back in from disk. Repeated transfer of data buffers back and forth between memory and disk is called thrashing, and because of the heavy I/O involved typically causes very poor performance. When the number of I/O reads exceeds the number of inserts by more than the margin allowed by the option HASH_THRASHING_PERCENT, the message noted above “Hash Insert thrashing detected” will be written to the .iqmsg log. The warning message “Hash Insert forced buffer unpinning” is intended as a precursor so that action can be taken to avoid the poor performance that comes with thrashing.

 

Buffers reused as indicated by this message are found only in that particular user connection’s allocation; buffers cannot be acquired from another user.

 

Factors involved in the warning

 

There are basically three factors involved in the appearance of the “Hash Insert forced buffer unpinning” warning in the .iqmsg log:

- how much IQ Temp Cache memory is allocated to the user

- how much IQ Temp Cache any one object holds (pins) in memory

- how much demand there is for IQ Temp Cache in the current activity

 

Each IQ user connection is allocated a portion of the overall amount of IQ Temporary Cache made available to SAP Sybase IQ when it boots. Typically the startup configuration parameter “-iqtc” is used to specify this. Of total IQ Temporary Cache, a user’s allocation is determined from the number of connected users, or the number of commands executing, which could be a somewhat greater number (due to commands running in cursors, for example). Various factors are applied in arriving at a basic allocation of buffers. A user connection makes use of buffers for several purposes including sorting, prefetching, hashing and internal arrays.

 

The amount of IQ Temporary Cache used to build a column’s FP index depends on the number of distinct values, the width of the column datatype, and other optimized FP indexes being used at the same time.

 

For any specific request building an FP index, the base buffer quota is the user’s allocation times the value of the option HASH_PINNABLE_CACHE_PERCENT. For example, if a user is allowed 1000 buffers and HASH_PINNABLE_CACHE_PERCENT is the default of 20 (20%), then a request for loading data for a column can pin 200 memory buffers as a maximum. If there are between 65536 and 16777216 distinct values, an FP(3) index is called for, and as the buffer quota gets used up the user could see the “Hash Insert forced buffer unpinning” warning message in the .iqmsg log.

 

A request will normally ask for all the buffers available in its quota. If they are free they are allocated. If not, available buffers up to the maximum are allocated. A hash structure for an FP(3) index can grow to twice the normal quota if there are plenty of available buffers according to an algorithm, but only by 10 buffers at a time.

 

Actions to take

 

The best option for preventing the occurrence of this warning message, and consequently avoiding possible progression to cache thrashing, is to significantly increase the amount of memory allocated to the IQ server for IQ Temporary Cache. This typically means increasing IQ Main Cache as well, since the IQ 15 Performance and Tuning Guide recommends maintaining a fixed ratio between the two cache allocations. A large FP(3) index will also require more space in IQ Main Cache.

 

The value of option HASH_PINNABLE_CACHE_PERCENT could be increased. This will allow any hash object to use more of the user’s IQ Temporary Cache allocation. The Reference manual topic for the HASH_PINNABLE_CACHE_PERCENT option advises this should be done only “with extreme caution” and after testing a wide variety of application scenarios. Allowing an operation for any one hash object to consume more of the buffer quota will leave less for other hash objects.

 

This warning is only logged when inserts are made to an FP(3) index. As noted, FP(3) indexes can become large. The server can be prevented from building FP(3) indexes beyond a certain size by limiting the values of both the FP_LOOKUP_SIZE and FP_LOOKUP_SIZE_PPM options. They can be set for an individual connection or for the Public group. When the server cannot build an FP(3) index and there are more distinct column values than can be accommodated by an FP(2) index, a “flat” FP index will be built. A flat FP does not use a lookup table, but keeps actual column values in the column array. Operational and performance tradeoffs between using FP(3) and flat-style FP indexes should be tested.

 

As an additional approach, it may be possible to manage the application load so there are fewer tasks demanding IQ Temporary Cache at one time.

Query Plan Changes- IQ 16

$
0
0

Query html plans are generated for load execution plans.  Load execution html plans are generated for these statements:

 

  • Alter table add column (with default value)
  • Create Index
  • Declare cursor for update
  • Insert..into
  • Insert..location
  • Insert..values
  • Load..into
  • Parallel IQ statements

 

To Generate html plans for load execution, you will need to set appropriate Query options:

 

  • set option public.query_plan='on'
  • set option public.query_detail='on'
  • set option public.query_plan_as_html='on'
  • set option public.query_plan_after_run='on'
  • set option public.query_timing='on'
  • set option public.query_plan_min_time='on'
  • set option public.query_plan_as_html_directory='on'

 

For detailed information on these options:

 

http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc00801.1600/doc/pdf/iqrefso.pdf

 

 

These option can be set as temporary option as well,  Html plans should be only used to evaluate the performance of a particular query or load. Query_plan database option shouldn't be set to ON as it can significantly impact performance, especially as the volume of insert..value statement increases.

 

Query Plan Nodes:

 

  • Root Node:  This node is created at very top of the Data Flow Tree.
  • Sequencer:  In the Data Flow Tree, this node has two or more childen.  Pass1 branch of load has Child1 and Pass2 branch of load  has Child2.
  • Parallel Combiner: This Node takes multiple streams of rows from below, and 1 stream of rows is produced.
  • Index Insert: Insert into indexes. Sorted data values used for HG or WD or TEXT are retrieved and inserted into index. Only unique constraint is applied during index insert.
  • RowID Generator: Unique rowid  for each row is generated.
  • Leaf:  This node is created at the bottom of the Data Flow Tree. 

 

New Query Plan Nodes:

  • Distinct Sort(inserter):  Node in Data Flow Tree indicating early aggregation algorithm being used.
  • Distinct Sort(retriever): Node in Data Flow Tree indicating early aggregation algorithm being used.
  • Grouping Sort(inserter): Node in Data Flow Tree indicating early aggregaton algorithm being used.
  • Grouping Sort(retriever):  Node in Data Flow Tree indicating early aggregation algorithm being used.
  • Join(Asymmetric Sort-Merge): Node in Data Flow Tree indicating asymmetric sort-merge join being used.
  • Join(Asymmetric Sort-Merge Pushdown): Node in Data Flow Tree indicating asymmetric sort-merge join being used.
  • Order By(Per-Work-Unit):  Node in Data Flow Tree indicating an operator that performs a sort on a work-unit worth of data. For example, it displays on the left branch under the  Asymmetric Sort-merge Join node.

Featured Content for SAP Sybase IQ

$
0
0

sm_sunflower.jpg

SAP Sybase IQ 16 News

Subscribe and receive the quarterly SAP Sybase Adaptive Server Enterprise (ASE) Newsletter. Each issue contains the latest information about ASE and Sybase IQ's product features, best practices, news articles, customer stories and upcoming event details. Subscribe today!

 

 

SAP Sybase IQ free downloads: Choose the right fit for you

Get actionable intelligence delivered at the speed of business – with SAP Sybase IQ analytics server. Transform the way you compete to win.

 

SAP Sybase IQ UDA: Full Text Search by Examples

$
0
0

The Unstructured Data Analytics (UDA) option extent the capabilities of SAP Sybase IQ to perform Data Mining.  Here you will find examples of queries that are possible using the UDA option.  You will need a UDA license to run these queries.

 

For these examples I created a table as follow:

 

CREATE TABLE DBA.MyReport  (

          MyRptID       integer NOT NULL,

          MyRptText      CLOB NULL,

          PRIMARY KEY(MyRptID)

  );

   

The table was populated with 1.7 millions of records, the MyRptText column was populated with clinical laboratory results saved in text format.  The database as created with CASE RESPECT.

 

Two (2) indexes and one(1) text configuration object were created, a text index using the default_char configuration object, and a NGRAM text index using a new text configuration object.

    

Create text index RptTextIdx on DBA.MyReport  

(MyRptText) configuration default_char;

     

-- Create a new configuration object

  

CREATE TEXT CONFIGURATION myFuzzyTextConfig FROM default_char;

 

-- Change to NGRAM and set maxlength to 5

 

ALTER TEXT CONFIGURATION myFuzzyTextConfig

  TERM BREAKER NGRAM;

 

ALTER TEXT CONFIGURATION myFuzzyTextConfig

  MAXIMUM TERM LENGTH 5;

 

CREATE TEXT INDEX myFuzzyTextIdx

ON DBA.MyReport (MyRptText) CONFIGURATION myFuzzyTextConfig;

    

All the queries were successfully executed, the results are not included in this document.  No performance data is provided, but all of the queries executed in 1 minute or less.

    

Use Case 1: Find the 10 most relevant records relative to a term using the calculated score from the CONTAINS clause.

 

select MyRptID, ct.score, MyRptText

  FROM MyReport CONTAINS (MyReport.MyRptText, 'Difficile | DIFFICILE') as ct

  where ct.score > 10

  order by ct.score desc;

    

Use Case 2: Search records containing a phrase (can also be done with LIKE)

 

select count(*)

  FROM MyReport

        CONTAINS (MyReport.MyRptText, 'Clostridium difficile toxin A/B DETECTED' ) as ct;

  

select MyRptID, MyRptText

  FROM MyReport

        CONTAINS (MyReport.MyRptText, 'Clostridium difficile toxin A/B DETECTED' ) as ct;

 

Use Case 3: Search records using Boolean expression

  

select *

  FROM MyReport

        CONTAINS (MyReport.MyRptText, '"Clostridium difficile toxin A/B DETECTED" and "ER"' ) as ct;

 

select *  

  FROM MyReport CONTAINS (MyReport.MyRptText,'(Clostridium | difficile | DIFFICILLE) AND DETECTED') as ct;

  

select *  

  FROM MyReport CONTAINS (MyReport.MyRptText,'((Clostridium | difficile | DIFFICILLE) AND DETECTED) AND NOT (" by PCR")') as ct;

  

select *  

  FROM MyReport CONTAINS (MyReport.MyRptText,'((Clostridium | difficile | DIFFICILLE) AND DETECTED) AND (" by PCR")') as ct;

    

Use case 4: Find terms in proximity

 

select

  FROM MyReport CONTAINS (MyReport.MyRptText,'difficile NEAR blood') as ct;

 

select

  FROM MyReport CONTAINS (MyReport.MyRptText,'difficile NEAR negative') as ct;

 

select

  FROM MyReport CONTAINS (MyReport.MyRptText,'cholesterol') as ct

  order by ct.score desc;

 

select

  FROM MyReport CONTAINS (MyReport.MyRptText,'gallbladder BEFORE cholesterol') as ct

  order by ct.score desc;

  

select MyRptID, ct.score, MyRptText 

  FROM MyReport CONTAINS (MyReport.MyRptText,'cholesterol NEAR ultrasound') as ct

order by ct.score desc;

  

select MyRptID, ct.score, MyRptText 

  FROM MyReport CONTAINS (MyReport.MyRptText,'cholesterol NEAR[2,5] ultrasound') as ct

  order by ct.score desc;

   

Use case 5: Find terms using prefixes

 

select count(*) 

  FROM MyReport CONTAINS (MyReport.MyRptText,'Urin*') as ct;

 

select count(*) 

  FROM MyReport CONTAINS (MyReport.MyRptText,'Urin* AND NOT Nega*') as ct;

  

select count(*) 

  FROM MyReport CONTAINS (MyReport.MyRptText,'Urin* AND Aty* AND NOT Nega* ') as ct;

 

select count(*) 

  FROM MyReport CONTAINS (MyReport.MyRptText,'Urin* AND Aty* AND NOT Nega* AND blood ') as ct;

 

select MyRptID, ct.score, MyRptText 

  FROM MyReport CONTAINS (MyReport.MyRptText,'Urin* AND Aty* AND NOT Nega* AND blood ') as ct

  order by ct.score desc;

 

Use case 6: Search for term similar to a given term (FUZZY SEARCH)

 

select count(*)   FROM MyReport  

        CONTAINS (MyReport.MyRptText, 'FUZZY "difficile"' ) as ct;

 

select count(*)   FROM MyReport  

        CONTAINS (MyReport.MyRptText, 'difficile AND FUZZY "DETECTED"' ) as ct;

  

select count(*)   FROM MyReport  

        CONTAINS (MyReport.MyRptText, 'difficile AND FUZZY "DETECTED" AND NOT DETECTED' ) as ct;

 

select top 100 FROM MyReport  

        CONTAINS (MyReport.MyRptText, 'difficile AND FUZZY "DETECTED" AND NOT DETECTED' ) as ct

  order by ct.score desc;

    

Use case 7:  Find the number of indexed values in which a term appears

 

select * from sa_text_index_vocab('RptTextIdx','MyReport','DBA')

   where term like '%[Cc]holesterol%';

  

select * from sa_text_index_vocab('RptTextIdx','MyReport','DBA')

   where term like '%[Cc][Hh][Oo][Ll][Ee][Ss][Tt][Ee][Rr][Oo][Ll]%';

 

select * from sa_text_index_vocab('RptTextIdx','MyReport','DBA')

   where term like '%[dD][Ii][Ff][Ff][Ii][Cc][Ii][Ll][Ee]%';

 

   

These are only a few examples of the type of queries that can be used on a CLOB column using the UDA option; This is not a complete list of possibilities.

SAP Sybase IQ 15.4 Manuals (PDF) to work with phone/tablet PDF readers like Aldiko

$
0
0

Many PDF readers for smart phones (Android/iphone) and tablets manage the pdf files Calibre_Logobased solely on the Title and Author fields in the PDF file. While for this is fine for your average book, it is not all that helpful with manuals that tend to have abbreviated or no data in the title/author fields. In the case of the manuals for Sybase IQ, I’m unable to load the manuals for say v15.4 and v16.0 as they have the same Title/Author data.

How to fix? Easy. Go get Calibre. Drop the PDF files on to the running Calibre. Edit them by hitting the E key.

In my case, I edited the “Title”, “Author”, “Tags”, “Publisher” and “Languages”:

Calibre

Calibre doesn’t modify the PDF files themselves so I will need to export the files to a custom directory. In Calibre nomenclature, this is “Saving”. Highlight all the titles you want to export and hit “S” twice. Why twice? No idea. Choose the directory.

You can now copy the exported PDF files to your phone, tablet, whatever without fear of the v16.0 version of the P&T Guide being rejected by Aldiko because the v15.4 version is already added.

Here are the IQ v15.4 manuals that I’ve ‘fixed’ to work with Aldiko. They are identical to the PDFs on sybooks with the exception of the PDF fields I mentioned previously.

No copyright infringement is intended. SAP/Sybase, please feel free to take these and host them or fix the PDF files on sybooks

http://froebe.net/blog/2013/08/30/sap-sybase-iq-15-4-manuals-fixed-to-work-with-tabletphone-pdf-readers-like-aldiko/

 

This document was generated from the following discussion: SAP Sybase IQ 15.4 Manuals (PDF) to work with phone/tablet PDF readers like Aldiko

Quick Setup Steps for Setting Python for SAP SYBASE IQ

$
0
0

sqlanydb is SQL Antywhere Python database interfaces for data access API for the Phython language. Steps listed below can be used to set up Phython for Sap Sybase IQ. The Python database API is very powerful programming languge, set of  methods used  by this API provides a consistent  database interface independent of the database being used.   For detailed information about Python Database specification v2.0,

 

 

http://www.python.org/dev/peps/pep-0249/.

 

and for supported platforms refer to:

 

http://www.sybase.com/detail?id=1068981.

 

From SAP Sybste Release directory,  cd $SYBASE/IQ-15_4

 

1. [IQ-15_4]$ source IQ-15_4.csh
Sourcing SYBASE.csh
2. [ IQ-15_4]$ cd sdk
[sdk]$ ls
c  dbcapi  include  perl  perlenv  php  phpenv  python  ruby
3. [sdk]$ cd python
[ python]$ ls
build  scripts  setup.py  sqlanydb.py

 

4. As root


[ python]$ sudo python setup.py install
[sudo] password for user1:
running install
running build
running build_py
running install_lib
copying build/lib/sqlanydb.py -> /usr/lib/python2.6/site-packages
byte-compiling /usr/lib/python2.6/site-packages/sqlanydb.py to sqlanydb.pyc
running install_egg_info
Writing /usr/lib/python2.6/site-packages/sqlanydb-1.0.2-py2.6.egg-info
[ python]$

 

5, edit test.py(This script will start IQ server )  with correct dbn and dbname

 

6. python /rhvm1_work/rel154_iq_esd2/IQ-15_4/sdk/python/scripts/test.py

===================================
IQ server starting with:
     10 connections         (       -gm )
     14 cmd resources       ( -iqgovern )
    136 threads             (     -iqmt )
    512 Kb thread stack size   (   -iqtss  )
  69632 Kb thread memory size ( -iqmt * -iqtss )
      2 IQ number of cpus  ( -iqnumbercpus )
      0 MB maximum size of IQMSG file ( -iqmsgsz )
      0 copies of IQMSG file archives ( -iqmsgnum )
=============================================================

Warning: Using default value for -iqmc or -iqtc is insufficient for most applications
Warning: Using default value for -iqmc or -iqtc is insufficient for most applications
Using licenses from:/usr/sybase/rel154_iq_esd2/SYSAM-2_0/licenses/keyserver.syb
ase.com.lic:/usr/sybase/rel154_iq_esd2/IQ-15_4/demo/simplex/*.lic
Checked out license for 2 IQ_CORE (2013.0331/31-mar-2013/11F0 02BD A308 E97B) will expire Mon 01 Apr 2013 12:00:00 AM PDT.
WARNING: IQ functionality that requires the IQ_CORE license will be disabled on
Mon 01 Apr 2013 12:00:00 AM PDT, unless a suitable IQ_CORE license is obtained before that date.
sqlanydb successfully installed.
[simplex]$ Checked in license for 2 IQ_CORE (2013.0331/31-mar-2013/
11F0 02BD A308 E97B).
Checked in license for 2 IQ_CORE (2013.0331/31-mar-2013/11F0 02BD A308 E97B).

[simplex]$

 

The sqlanydb is now ready to use.


SAP Sybase IQ 16.0 Hardware Sizing Guide

$
0
0

This document will attempt to highlight the main areas of concern for sizing a Sybase IQ environment. Sizing of SAP Sybase IQ is generally confined to CPU, memory, and storage.  It should also be noted that this document is a work in progress and most points contained in it come from real world experience. Should this document be followed it is at the sole responsibility and discretion of the reader and implementation team.

 

http://scn.sap.com/docs/DOC-41455

Restore Coordinator On Different Location

$
0
0

Following are the steps to be followed,  for restoring IQ 15.x and  16.x  MPX coordinator server on different host machine:

 

 

  1. Take  full backup of  the Cordinator server wth DBA authority.

  

  1> backup database to /usr/sybase/local/mpx/iqdemo.dmp'

  2>   go

 

  

Note: Database can also be backup using mulitiple stripes, For complete syntax check

 

   http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01759.1601/doc/pdf/iqbackup.pdf

http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc00801.1510/pdf/iqrefso.pdf

  

2. To restore own all the servers (coordinator and all econdary nodes) in multiplex.  Make sure backup   files   are  moved to  another location

in binary mode.

 

   Note: If restoring backup from production coordinator to DR site, and it is not possible to shutdown MPX on production, then on DR host, utility_db can be
started with coordinator server name(production coordinator server name) with startup parameter, DoBroadcast=NO

 

  4.  If restoring to different location, it is  better  to have directory structure  same on target site as  on the source site, if not,  then create a symbolic link  with exactly same name as source on target host.

  

For Example

 

Source Directory
(iq154w)

 

 

/usr/sybase/rel154_iq/IQ-15_4/demo/mpx

 

   Temp files for IQ_SYSTEM_TEMP:

  

/usr/sybase/rel154_iq/IQ-15_4/demo/tempdev

 

  

Target restore directory:

 

 

   /usr/sybase/rel154_iq/IQ-15_4/demo/mpx6

 

  

Target restore temp file directory:

 

  

/usr/sybase/rel154_iq/IQ-15_4/demo/mydev

 

 

Setup Symbolic links:

 

   ln -s /usr/sybase/rel154_iq/IQ-15_4/demo/mpx6 /usr/sybase/rel154_iq/IQ-15_4/demo/mpx

 

 

ln -s /usr/sybase/rel154_iq/IQ-15_4/demo/mydev /usr/sybase/rel154_iq/IQ-15_4/demo/tempdev

 

 

4.Change  to restore directory

 

    sybase@myhost mpx]$ pwd

    /usr/sybase/rel154_iq/IQ-15_4/demo/mpx6

  

5. Start the utility_db using the coordinator server name or new server name

 

  

[sybase@myhost mpx]$ cat iqrestore.cfg

 

# iqdemo.cfg

#
------------------------------------------------------------

  # Default startup parameters for the ASIQ demo database

  # ------------------------------------------------------------

  

-n  iq154w

-x  tcpip{port=8999}

  

# The following parameters are also found in the configuration file

# /usr/sybase/rel154_iq/IQ-15_4/scripts/default.cfg.  Any parameters not specified below

# and not in the start up parameter list, will be added by start_iq

# using default.cfg as a guide.

   -c  48m

  -gc 20

  -gd all

  -gl all

  -gm 10

  -gp 4096

  -iqmc 100

  -iqtc 100

 

 

 

6. connect to utility_db and run restore command 

 

[sybase@myhost mpx]$ cat restore.sql

 

restore database '/usr/sybase/rel154_iq/IQ-15_4/demo/mpx6/iqdemo.db'

  from '/redeye2/sbagai/rel154_iq/IQ-15_4/demo/mpx6/iqdemo.dmp'

  ;

  

[sybase@redeye mpx]$ dbisql -c 'uid=DBA;pwd=sql;eng=iq154w;dbn=utility_db' -host myhost -port 8999 -nogui restore.sql

 

Execution time: 23.485 seconds

 

[sybase@mynost mpx]$

 

 

7. Shutdown utility_db

 

[sybase@redeye mpx]$ stop_iq

 

8. Make sure dbfile  for IQ_SYSTEM_TEMP exist as before and or files of the correct size, check with cksum , if you have issues with temporary dbfile,  then start
coordinator with –iqnotemp <size> switch in addition to –iqmpx_sn 1 and iqmpx_ov 1 switches.

 

9.  Start coordinator server with single node(–iqmpx_sn 1) and override (iqmpx_ov 1 )switches

 

  [sybase@myhost mpx]$ cat iqdemoov.cfg

 

# iqdemo.cfg

#
------------------------------------------------------------

# Default startup parameters for the ASIQ demo database

#-----------------------------------------------------------

-n  iq154w

-x  tcpip{port=8999}

  

# The following parameters are also found in the configuration file

# /usr/sybase/rel154_iq/IQ-15_4/scripts/default.cfg.  Any parameters not specified below

# and not in the start up parameter list, will be added by start_iq

# using default.cfg as a guide.

  -c  48m

  -gc 20

  -gd all

  -gl all

  -gm 10

  -gp 4096

  -iqmc 100

  -iqtc 100

-iqmpx_sn 1

  -iqmpx_ov 1

 

  [sybase@myhost mpx]$ start_iq @iqdemoov.cfg iqdemo.db

 

10.  Drop all the secondary nodes, using drop multiplex server statement

  

[sybase@myhost  mpx]$ iisql -UDBA -Psql -Siq154w

  1> drop multiplex server iq154q

  2> go

 

11. Once last secondary node is dropped, coordinator shuts down automatically,  signifying conversion to simplex.

 

12.  Restart the coordinator without single node or override switches(-iqmpx_sn 1, -iqmpx_ov)

    

        [sybase@myhost mpx]$ start_iq @iqdemo.cfg iqdemo.db

 

13.  Recreate all the secondary nodes with the correct location path manually/ Sybase central/Sybase Control Center

 

  [sybase@myhost q1]$ isql -UDBA -Psql -Siq154w

 

     1> create multiplex server iq154q

           2> database '/usr/sybase/rel154_iq/IQ-15_4/demo/mpx6/q1/iqdemo.db'

           3> host 'myhost' port 9001

           4> Role WRITER STATUS INCLUDED

       5> go

   After first secondary node is created, the coordinator server automatically shuts down, signifying conversion to multiplex.

 

   Note: In “create multiplex server” command you must provide database file extension .db

 

14.  Restart the coordinator server

 

    [sybase@redeye mpx]$ start_iq @iqdemo.cfg iqdemo.db

 

   When Coordinator server is started you will see warning, this warning can be ignored, because IQ_SYSTEM_TEMP dbspace doesn’t contain files, This is true for
all secondary nodes created in this step.

  

        I. 01/06 19:46:36. Finished checkpoint of iqdemo" (iqdemo.db) at Wed Jan 06 2012 19:46

        I. 01/06 19:46:38. Database "iqdemo" (iqdemo.db) started at Wed Jan 06 2012 19:46

        I. 01/06 19:46:38. IQ Multiplex Coordinator Server iq154w.

        I. 01/06 19:46:38. Database server started at Wed Jan 06 2012 19:46

        I. 01/06 19:46:38. Trying to start SharedMemory link ...

        I. 01/06 19:46:38.     SharedMemory link started successfully

        I. 01/06 19:46:38. Trying to start TCPIP link
        ...

        I. 01/06 19:46:38. WARNING : Multiplex environment incorrect for this server

        I. 01/06 19:46:38. Please connect and run procedure sp_iqmpxvalidate for help

        I. 01/06 19:46:43.     TCPIP link started successfully

        I. 01/06 19:46:43. Now accepting requests

      

                 New process id is 15948

                Server started successfully

 

15. synchronize secondary nodes(if you checked ‘Generate Admin Scripts’ when creating the mpx server in Sybase central, then you can run sync_server script instead of dbbackup

 

             /usr/sybase/rel154_iq/IQ-15_4/demo/mpxbk

           [sybase@myhost mpxbk]$ cd q1

           [sybase@myhost q1]$ ./sync_server

           SQL Anywhere Backup Utility Version 12.0.1.6567

        (724 of 722 pages, 100% complete)

           Database backup completed

          SQL Anywhere Transaction Log Utility Version 12.0.1.6537

         "/myhost/sybase/rel154_iq/IQ-15_4/demo/mpx/q1/iqdemo.db" was using log file "iqdemo.log"

          Transaction log filename not changed

        "/myhost/sybase/rel154_iq/IQ-15_4/demo/mpx/q1/iqdemo.db" was using no log mirror file

         Transaction log mirror filename not changed

         Transaction log starting offset is 0001060521

     Transaction log current relative offset is 0000006233

      Or   you can use following steps to synchronize secondary nodes:

 

    Chapter 7 : Backing Up and Restoring Data in a Multiplex Environment

    

       http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01839.1601/doc/pdf/iqmpx.pdf

 

              a.  Remove the .LOG transaction log file in the secondary server directory.
             b. Back up the catalog store to   the directory where the secondary server runs:
c.  dbbackup -y -d -c "uid=dba;pwd=sql;links=tcpip{port=2763};eng=mpxnode_c" /host1/mpx/

             d. Set the log file path:

e.  dblog -r -t mpxtest.log mpxtest.db 
             f. Start the secondary node using  the start_iq command.

             g. Repeat the previous steps for   each secondary server in the multiplex.

 

  16.  Start secondary nodes

    [sybase@myhost q1]$ start_iq @iqdemo.cfg iqdemo.db

 

17. Connect to each secondary node and add files to IQ_SYSTEM_TEMP

 

  [sybase@myhost  q1]$ isql -UDBA -Psql -Siq154q

 

   1> alter dbspace IQ_SYSTEM_TEMP ADD FILE
iqdemotmp '/myhost/sybase/rel154_iq/IQ-15_4/demo/mpx6/q1/iqdemotmp' size 300

   2> go

 

18. Connect to coordinator server and run sp_iqmpxvalidate, it should report ‘no error detected’

   

    [sybase@myhost q1]$ isql -UDBA -Psql -Siq154w

                      1> exec sp_iqmpxvalidate

                     2> go

                  Messages

 

             -------------------------------------------------------------

                No errors detected

  

       (1 row affected)

 

              (return status = 0)

 

  19. Optional: If required, incremental backups from source coordinator server can be restored on target(DR) coordinator server(To restore using incremental backup follow step 2-18)

     

 

NOTE: Sometimes if after restore, before file has been added to secondary node, you try to create local temporary table, on coordinator node,  you will get
error message that IQ_SYSTEM_TEMP dbspace does not have dbfiles.

 

    1> DECLARE LOCAL TEMPORARY TABLE temp(col1 int)

    2> go

    Msg 21, Level 23, State 0:

   SQL Anywhere Error -1009128: The IQ_SYSTEM_TEMP dbspace does not have dbfiles for this server.

  -- (s_blockmap.cxx 3589)

 

   In that case, follow steps from 17-18

 

 

 

Forced Recovery Steps for Coordinator

$
0
0

if you are unable to start SAP IQ  Coordinator Server or  Database  in multiplex system,  check .iqmsg and .srvlog for diagnostics information, depending

on the error you might be able to recover Coordinator using  forced recovery steps. Most of the time assertion failed  error messages are SA errors. Forced

recovery will not work for SA assertion failed  error messages, you will need to restore from backups. It is advisable to contact  SAP Sybase

technical support for assistance. For IQ  issues (s_buf, freelist),  you can follow steps listed below for performing  forced recovery. For any

questions or concerns contact SAP Sybase Technical support.

 

Before using forced recovery, it is very important to  restrict access to the database, you can either use -gm 1(allows single connection plus

one DBA connection so that DBA can connect and drop others in case of emergency, or  issue sa_server_option('disable_connections', 'on')

after starting server in forced recovery mode, after forced recovery procedures, issue sa_server_option('disable_connections', 'off') on the same

connection.

 

Steps:

 

1. Shut down all  secondary nodes using stop_iq or stop_server.sh

2. Start Coordinator with -iqfrec , -iqmpx_sn 1 , -gn 1  and -gd 1

 

       start_iq -n <coordinator_server>  @params.cfg  -gd dba -gm 1 -iqfrec <coord db name without .db extn> -iqmpx_sn 1  <coord db name with .db extn>

 

      start_iq  -n < coordinator_server>  @params.cfg  -gd dba  -gm 1 -iqfrec  coord -iqmpx_sn 1 coord.db

 

3. Forced recovery marks all allocation within database as in use, run sp_iqcheckdb in  dropleaks mode to reset allocation map to correct

state.

            set temporary option dbcc_log_progress='on' ;

            sp_iqcheckdb 'dropleaks database';

 

    If there are no errors and sp_checkdb displays message 'Freelist Updated'  you have recovered leaked blocks.  Continue to Step 4

 

   if incosistency is found, follow the instructions for dropping inconsistent indexs/tables or columns to drop inconsistent objects in Administration:

   Backup, Restore and Data Recovery (Chapter: System Recovery and Database Repair)

 

             http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01759.1601/doc/pdf/iqbackup.pdf

 

4.  Issue Checkpoint

5. Shutdown Coordinator using stop_iq or stop_server.sh script

6.  Restart Coordinator normally without -iqfrec, -iqmpx_sn 1 , -gd 1 and -gm 1 switches

 

          ./start_server.sh

 

      Note:  start_server.sh and stop_server.sh scripts are generated by Sybase Control Center in  Server .db directory.

 

7.  If corruption was in IQ_SYSTEM_MAIN then run_sp_iqcheckdb on IQ_SYSTEM_MAIN else run dbcc on entire database.

 

           set temporary option dbcc_log_progress='on' ;

           sp_iqcheckdb 'verify dbspace IQ_SYSTEM_MAIN';

           sp_iqcheckdb 'verify database'

 

8.  Synchronize Secondary Nodes.

MPX(IQ Multiplex): How to Create DR without adding new temp dbfiles.

$
0
0

From SAP Sybase IQ 15.2 esd1 and above, Restore to DR side can be done by starting utility_db with DR Coordinator Server Name,  it no longer have to same

as the production Server. And if directory structure and dbfile name and sizes are exactly  same as production Coordinator Srer, then DBA do not need to drop

existing Secondary Servers.  For complete restore instructions, please see:

 

http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc01839.1601/doc/html/san1278444472827.html

 

After restoring DR Coordinator, DBA can start DR Coordinator with -iqmpx_sn 1 -iqmpx_ov 1 switch and issue commands listed below to alter

DR configuration:

 

(DBA)> alter multiplex server PRDIQC  rename DRIQC host 'myhost' port 9981

 

  

(DBA)> alter multiplex server PRDIQS rename DRIQS host 'myhost' port 9982

 

   Restart Multiplex normally without -iqmpx_sn 1 and -iqmpx_ov 1 switches.

  PRDIQC  -> Production coordinator name

  PRDIQC -> Production secondary node

 

  DRIQC  -> New DR coordinator name

  DRIQS ->New DR secondary node

Stopping/Excluding Secondary Nodes In SAP Sybase Multiplex

$
0
0

1. Secondary nodes shouldn't excluded while they are running prior to Coordinator shudown.  It is preferable to shut them down, using dbstop instead, then exclude them.

 

       1a. It is not necessary to exclude Secondary nodes before Coordinator have been shutdown, but may speed up Coordinator shutdown by shortening the checkpoint  log.

 

       1b. You only need to exclude Secondary nodes, only if they are to be taken down offline for an extended amount of time while the Coordinatoris running, allowing the Coordinator to disregard version references of these nodes when doing global version cleanup.

 

       1c. If you need to exclude Secondary node for any reason, shutdown the Secondary node first and then login to Coordinator and  then exclude it.

 

2.  After shutting down the Secondary nodes, wait  a couuple of minutes and then issue checkpoint twice.

 

       2a. This time period allows for version collection to start and reduce number of active versions which needed to be checkpointed at shutdown.

 

3. Avoid using kill -9, it is preferable to use dbstop with -y option.

 

        3a.  Killing the server with -9  forces the Coordinator in crash recovery mode.

 

 

http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc00168.1601/doc/html/san1276279407063.html

IQ 16: Configure ASE 12.5.x Remote Server

$
0
0

When migrating from SAP Sybase IQ version 15.2 to version 16,
if you had configured remote servers to SAP Sybase ASE 15.2.x using JDBC, you
need to drop and recreate the remote servers using ODBC. 

 

 

Test Environment

    

  • SAP Sybase IQ 16 SP#1 on Solaris 10 64b running
    on Sun SPARC 64-bit.
  • SAP Sybase ASE 12.5.3 ESD#7 on Solaris 8 64b
    running on Sun SPARC 64-bit.

 

     

Procedure Overview

   

 

  1. Install and configure Sybase ODBC driver on the
    SAP Sybase IQ server.
  2. Upgrade the DMA scripts on the ASE server.
  3. Configure the ODBC Data Source Name.
  4. Drop and recreate the remote server in SAP
    Sybase IQ.

 

 

 

The detailed procedure can be see here http://arbime.wordpress.com/2013/10/25/iq-16-configure-ase-12-5-x-remote-server/

Creating ASEODBC Connection from SAP Sybase IQ to SAP Sybase IQ Adaptive Enterprise Server

$
0
0

Steps listed below can be used to setup remote connection from SAP Sybase IQ and SAP Sybase Adaptive Server Enterprise. First you will need to install  OCS 157 64 bit  on IQHOST in diferent location for using ASE odbc driver

 

 

 

1. setup Environment variables.

 

cd to $SYBASE/IQ-15_4 and edit IQ-15_4.csh and add:

 

   setenv LIBPATH "${LIBPATH}:/usr/sybase/rel157/DataAccess64/ODBC/lib"

   setenv PATH "${PATH}:/usr/sybase/rel157/DataAccess64/ODBC/lib"

  

setenv ODBCINI /usr/sybase/rel154_iq_esd3/odbc.ini

  setenv ODBCHOME /usr/sybase/rel154_iq_esd3/

 

   2. source IQ-15_4.csh

 

   3. Add SAP Adaptive Server Enterprise and SAP Sybase IQ Server entries in interfaces

  

   % cat $SYBASE/interfaces

        aixase157

          master tcp ether ASEHOST 8412

          query tcp ether ASEHOST 8412

  

   iq154

          master tcp ether IQHOST 9522

          query tcp ether IQHOST 9522

 

   4. % cat $SYBASE/odbc.ini

   [aixase157]

  Description = Sybase ODBC Data Source

  UserID = sa

  Password =

  Driver = /usr/sybase/rel157/DataAccess64/ODBC/lib/libsybdrvodb.so

  Server = ASEHOST

  Port =  8412

  Database = pubs2

  UseCursor = 1

 

   [iq154]

  DRIVER=/usr/sybase/rel154_iq_esd3/IQ-15_4/lib64/libdbodbc12.so

  EngineName=iq154

  CommLinks=tcpip(host=IQHOST;port=9522)

  Userid=DBA

  Password=sql

  AutoStop=no

  DatabaseName=iqdemo

  

5. stop and restart IQ server

  6. Create Server and externlogin

     % dbisql -c 'dsn=iq154' -nogui

   (DBA)> select @@version

        @@version

 

  --------------------------------------------------------------------------------

Sybase IQ/15.4.0.3027/130814/P/ESD 3.6/Enterprise Linux64 - x86_64 - 2.6.18-194.el5/64bit/2013-08-14 03:34:24

(1 rows)

 

   Execution time: 0.014 seconds

 

  (DBA)> create server aixase157 class 'ASEODBC' usin

'Description=Sybase ODBC Data Source;Driver=/indigo61/sbagai/rel157/DataAccess64/ODBC/lib/libsybdrvodb.so;

UserID=sa,Password=;Server=indigo6;Port=8412;Database=pubs2;UseCursor=1'
;

  Execution time: 0.036 seconds

  

(DBA)> create externlogin DBA to aixase157
remote login 'sa' ;

 

Execution time: 0.032 seconds

  

(DBA)> forward to aixase157

 

0 row(s) affected

Execution time: 0.01 seconds

  

(DBA)> select @@version

 

expr_1

  --------------------------------------------------------------------------------

---------------

 

Adaptive Server Enterprise/15.7.0/EBF 19803SMP ESD#01 /P/RS6000/AIX 6.1/aseasap

 

/2918/64-bit/FBO/Wed Feb  8 06:09:12 2012

 

 

(1 rows)

  

Execution time: 0.85 seconds

  

(DBA)>

 

 

 


Setting Remote Procedure Call

$
0
0

This doc talk about how to set up RPC to other server, when RPC is  embedded with in begin … end block.   When Remote procedure is
created, engine has no knowledge of what Remote Procedure does and therefore It cannot implicitly define result set for Remote Procedure.

 

 

  If Remote Procedure is executed using  "call foo(...)" in DBISQL or dbisqlc, the application will open a cursor on the call statement
and the proper "cursor describe stuff" will automatically happen. BUT, if you attempt to execute a "call foo(...)" within a procedure
or batch, the engine does not automatically open a cursor on the call. Instead, it is up to the user to explicitly force a cursor to be opened either by
declaring a cursor within the stored procedure or batch or using the RPC in the from clause of a select statement. Furthermore, if the RPC is going

to be used in the from clause, then the RPC *MUST* be created with a RESULT clause in order to allow the engine to predefine the schema

of the expected result set.

 

 

STEPS:

 

Add source and target SAP Sybase IQ server entries in the interfaces. Add $ODBCINI and $ODBCHOME environment variables in $SYBASE/IQ-15_4/IQ-15_4.csh  and then source to setup environment variables

  1. Create remote server

  CREATE SERVER "iqdemo1543" CLASS 'SAODBC' USING 'dsn=iqdemo1543';

 

 

   2. Create  Externlogin

 

       CREATE EXTERNLOGIN "DBA" TO "iqdemo1543" REMOTE LOGIN "DBA" IDENTIFIED BY 'sql';

 

 

   3. Create Remote Procedure:

 

   create "DBA"."remwho"()

       result(

      ConnHandle unsigned bigint,

     IQconnID unsigned bigint,
 
     Userid varchar(255),

    ReqType varchar(255),

    IQCmdType char(32),

    BlockedOn unsigned bigint,

    BlockUserid varchar(255),

   IQCursors unsigned integer,

   IQThreads unsigned integer,

   IQIdle integer,

   SAIdle integer,

   TempTableSpaceKB unsigned bigint,

   TempWorkSpaceKB unsigned bigint )

   at 'iqdemo1543...sp_iqwho'

 

   4. Execute Remote Procedure

 


begin select * from remwho() end

ConnHandle           IQconnID             Userid
                                                        
ReqType

IQCmdType                        BlockedOn            BlockUserid



                                                                IQCursors

IQThreads  IQIdle      SAIdle      TempTableSpaceKB     TempWorkSpaceKB

--------------------------------------------------------------------------------

-----------------------------------------------------------------------

23                   5625                 DBA
                                                        
OPEN

IQUTILITYOPENCURSOR              0                    (NULL)
                                                             
0

1

      0           0           0                    0



(1 rows)


Execution time: 0.025 seconds

SAP IQ 16

$
0
0

The ability to draw intelligence from all potentially valuable information is a top priority, yet organizations are not prepared for rapidly increasing volumes and variety of data. Insufficient or incomplete data can result in missed opportunities or lost revenues. IT organizations need new approaches for unlocking the value of both existing and new sources of data in order to help their organizations remain competitive in today's data driven business climate. SAP IQ makes it simpler and more cost-effective to exploit the value of massive amounts of data at the speed of business, helping to transform companies through new insights that were previously impossible or impractical.

View this Document

Tableau Desktop Professional Edition 7.0 with SAP IQ 15.4, Report 379, May 2012

RLV Memory Management and Monitoring

$
0
0


In Memory row-level feature is available in SAP IQ 16.  This feature allows more than one user to update same table concurrenlty as long as they are adding or modifying different rows. You will need to configure IQ server with additional memory for RLV store ie. in addition to main cache, temp cache adn size of large memory pool, you will need need additional memory for RLV store.

 

RLV memory can be configured wtih startup switch -iqrlvmem (default 2G).

 

# iqdemo.cfg

# ------------------------------------------------------------

# Default startup parameters for the ASIQ demo database

# ------------------------------------------------------------

-n  iq1602

-x  tcpip{port=9447}

# The following parameters are also found in the configuration file

# /usr/sybase/rel16_iq_sp02/IQ-16_0/scripts/default.cfg.  Any parameters not specified below

# and not in the start up parameter list, will be added by start_iq

# using default.cfg as a guide.

-c  512m

-gc 20

-gd all

-gl all

-gm 10

-gp 4096

-iqmc 2000

-iqtc 2000

-zr sql

-iqrlvmem 4096

 

 

sa_server_option SA system procedure can be used to increase RLV memory dynamically:

 

   sa_server_option 'RLV_memory_mb', 8192

 

  This will configure RLV store size to 8192MB

 

  RLV memory usage depends on several factors:

 

  •   Table Schema(number of fixed and variable length columns and width of the columns)
  •   Concurrent loads and amount of data being loaded
  •    Number of cores
  •    Merge frequency

 

Every single RLV enabled table will require an in-memory table to hold its data. The memory requirement of this RLV store will change during the life of the table while it is RLV enabled. The initial size of RLV store needed during first DML can be calculated approximately using following formula:

 

 

 

#COLS = #columns + 1 (pseudo column)

Base RLVstore (BRS) = ( Initial setup of the RLV store )

                BRS ~= (#COLS)  x 16MB

 

Fixed datatypes of a subfragment uses default 16MB to store data in the RLV store. Variable datatypes uses default 512KB and bit datatypes uses 128KB default size.

 

    Example:

             Number of Cores=4

             Number of Columns=303

             RLV memory can be approximately calculated using:

                  16mb*4*303=~19GN

             for Singleton insert using only 1 core,  RLV memory used:

                   16mb *1*303= ~4.8GB

 

     Data is store in table fragment. Each table fragments is further subdivided into subfragments. Multiple table versions of the table fragments may be required to handle multiple transactions. Each table version will have the RLV memory size requirement of at least equal to this BRS(Base RLV Store). Multiple transactions will create multiple table store versions that need to be stored in memory simultaneously. Multiple table versions can quickly increase the memory requirement of the RLV enabled table. Old transactions can hold onto old references to the in-memory store whch will also consume memory.

 

  RLV Out Of Memory Errors:

  

Exception Thrown from rvlib/rv_TabStoren.cxx:178, Err#5, tid 3227 origtid 3227

O/S Err#: 0,ErrID: 15008 (rv_PersistenceException); SQLCode: -1013129, SQLState: 'QRLV1',Severity: 14

[22159]: RLVStore has run out of memory Exception Thrown from rvlib/rv_TabStoren.cxx:178, Err# 5, tid 3 origtid 3227

O/S Err#: 0, ErrID: 15008 (rv_PersistenceException); SQLCode: -1013129, SQLState: 'QRLV1', Severity: 14

 

   Explanation A:

              - To much data is being modified/ added by RLV transactions at once  

  Fix:
           -  Reduce the size of update/inserts and increase frequency of commits to  keep amount of data modified/transaction.

 

  Explanation B: 
          - Automated merge is not able to keep up with RLV workload

  Fix:  
           - Review RLV merger  history table(SYSIQRLVMERGEHISTORY)
    
               - Adjust the automated merger period(RV_AUTO_MERGE_EVAL_INTERVAL)

               -  Adjust automated thresholds, RV_MERGE_TABLE_NUMROWS,     RV_MERGE_TABLE_MEMPERCENT, and RV_MERGE_NODE_MEMSIZE  RLV Options.

 

    

   Explanation:

                   - Maximum RLV memory configure too low.

                   -increase maximum RLV memory using server startup switch –iqrlvmem

                  or

                   -increase maximum RLV memory using sa_server_option

    
                        sa_server_option  ‘rlv_memory_mb’, <max MB>

                  - issue manual merge to free of RLV memory using

      
                      sp_iqmergerlvstore (merge_type, table_name, [table_owner])

 

Note: if low or out of memory , then issue blocking merge, as non-blocking merge needs to setup additional  in-memory table fragments, which won’t be
possible,if already low/out of memory

  

Monitor:

  •     Use sp_iqrlvmemory system stored procedure to monitor RLV memory usage per table.

       

   (DBA)> sp_iqrlvmemory


    table_id  fragments      total       data dictionary     bitmap

    -----------------------------------------------------------------

      1089          1          6          0          3          2

       822          1         64         64          1          1


   (2 rows)

 

  •     Use sp_iqstatus system stored  procedure to monitor memory used by the RLV store

  

    (DBA)> select * from sp_iqstatus() where name like '%RLV%'

    Name


                Value

    ----------------------------------------------------------------


    RLV memory limit:


                4096Mb

    RLV memory used:

 

                76Mb

 

   (2 rows)

Login Redirection

$
0
0

Login Redirection feature in SAP IQ 16 allows load balancing by redirecting login connecting to overloaded node to less node in the same logical server. Login Redirection can be controlled by using login server policy options:

 

                    - LOGIN_REDIRECTION

                    - REDIRECTION_WAITERS_THRESHOLD

 

For information related to these options see:

 

 

                 http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc01839.1602/doc/html/jan1292624444594.html

 

           

                 http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc01839.1602/doc/html/jan1328621506262.html

             

Steps to setup Login Redirection:


In my setup, I have iq1602c, iq1602q1, iq1602q2, and iq1602q1 and iq1602q2 are members of logical server ls1

 

[sybase@myhost]$ dbisql -c 'dsn= iq1602c' -nogui

 

(DBA)> CREATE LOGICAL SERVER ls1 MEMBERSHIP (iq1602q1,iq1602q2);

 

  Execution time: 0.062 seconds

  (DBA)> alter ls policy root TEMP_DATA_IN_SHARED_TEMP=ON login_redirection = On dqp_enabled = 1 redirection_waiters_threshold = 5

   Execution time: 0.106 seconds

 

[sybase@myhost]$ dbisql -c 'dsn= iq1602c' -nogui

   

(DBA)> alter login policy root add logical server ls1

  Execution time: 0.059 seconds

 

we have iq1602q1 and iq1602q2 in  logical server ls1.  Now if we connect to Coordinator iq1602c,  login will be redirected to either iq1602q1
or iq1602q2.

   

dbisql -c 'uid=DBA;pwd=sql;eng= iq1602c;LS=ls1;NODETYPE=WRITER' -nogui

   

(DBA)> select @@servername

 

@@servername

 

    --------------------------------------------------------------------------------

 

iq1602q1

 

(1 rows)

   Execution time: 0.043 seconds

 

    Now for testing shutdown iq1602q1,  connection will be redirected to iq1602q2

   

dbisql -c 'uid=DBA;pwd=sql;eng= iq1602c;LS=ls1;NODETYPE=WRITER' -nogui

   

(DBA)> select @@servername

 

@@servername

     --------------------------------------------------------------------------------

 

iq1602q2


if connection redirection is enabled and login/connection  will redirect connections when:

 

  • ŸIf the connection is being made to a server which is not member of thespecified logical server
  • ŸIf there are more waiters at the gate than the specified threshold of waiters in the -iqgovern queue.
  • ŸIf the specified node type does have role different than that requested.

 

 

 

 

 

http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc01839.1602/doc/html/jan1328621506262.html

Viewing all 123 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>