Quantcast
Channel: SCN : Document List - SAP IQ
Viewing all 123 articles
Browse latest View live

IQ Shared Temp

$
0
0

IQ Shared Temporary Store Shared dbspace is available from  SAP IQ 15.3 and above. IQ Shared Temporary Store is automatically

created when new database is created or database is upgraded to IQ 15.3 or above. Newly created IQ_SHARED_TEMP does not

include any dbfiles.  Administrator will need to add dbfiles to IQ_SHARED_TEMP  using:

 

                           ‘Alter dbspace  IQ_SHARED_TEMP add file’

 

Shared raw device storage is required for IQ Shared Temporary Store, which should be accessible  and writeable by all Nodes in
the IQ Multiplex. IQ Shared Temporary Store(IQ_SHARED_TEMP) is used to store intermediate results between IQ Multiplex Nodes

during Distributed Query Processing(DQP).

 

The IQ Shared System Temporary (IQ_SHARED_TEMP) dbspace cannot be dropped. The last  dbfile in the IQ_SHARED_TEMP
dbspace can be dropped or made in READONLY in the multiplex single node mode(-iqmpx_sn 1). The IQ_SHARED_TEMP dbspace

cannot be set to READONLY. Temporary user tables cannot be created in IQ_SHARED_TEMP using the ‘IN IQ_SHARED_TEMP’
clause.

  • To add dbfiles to the Shared Temporary Store:

 

                         alter dbspace IQ_SHARED_TEMP add file <logical file name> ‘<physical file path>’

 

  • To drop dbfiles from the Shared Temporary Store:

 

                         alter dbspace IQ_SHARED_TEMP drop file <logical file name>

 

  • To alter Shared Temporary Store dbfile read/write status:

 

      alter dbspace IQ_SHARED_TEMP alter file <logical file name> [READONLY | READWRITE]

 

For complete syntax see:

 

                 http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc00801.1602/doc/html/san1281564739089.html

 

IQ Shared Temporary dbspace dbfiles are added as READONLY file status Once dbfile is added to IQ Shared Temporary dbspace,

all Secondary nodes in the multiplex attempt to access the dbfile and send the file status to the Coordinator. Once all Secondary Nodes

send valid file status to the Coordinator, dbfile can be altered to READWRITE file status and can be used for DQP. Attempt to alter file status to  READWRITE without a VALID file status from all Multiplex Secondary Nodes will result in an SQL error.

 

         
(DBA)> alter dbspace IQ_SHARED_TEMP alter  file iqsharedtmpdsp2 READWRITE

     
      Could not execute statement.

    
      File may not be accessible from one or more multiplex servers. Use   sp_iqmpxfilestatus for further diagnosis.

  
      -- (slib/s_db.cxx 3026)

  
     SQLCODE=-1009192, ODBC 3 State="HY000"

 

   Line 1, column 1   Press ENTER to continue...

 

The sp_iqmpxfilestatus stored procedure can be used to check file status. If executed on Coordinator node, it displays file status for Coordinator and for
every shared dbspace dbfile on every included Secondary Node.  If executed on Secondary Node, it only displays file status for only that Node.

 

  • File Status States:

            - VALID:  file path access and permissions are correct

            - INVALID_PATH: file path name access problem

            - INVALID_PERM – Operating System file permissions are incorrect.

 

In certain scenarios when one of the Secondary Node is down,  DBA can bring IQ SHARED TEMPORARY dbspace dbfile online by bypassing

file status check with the  FORCE READWRITE clause


           alter dbspace IQ_SHARED_TEMP alter file <logical file name> FORCE READWRITE


SAPIQ database options MAX_TEMP_SPACE_PER_CONNECTION and QUERY_TEMP_SPACE_LIMIT limits the sum of local and shared temporary space used by a user connection.

  • If the total of local and shared temporary space exceeds the quota set for user connection, DQP query will fail with “Temporary space quota

        exceeded” error.

  • In case for DQP query, temporary space used is total temporary space used by all nodes participating in the DQP query.
  • The IQ Shared Temporary Store(IQ_SHARED_TEMP) and Local temporary Store(IQ_SYSTEM_TEMP) uses temporary cache configured

        using –iqtc startup switch

 

IQ 16 allows temporary data in IQ_SHARED_TEMP from both temporary and global temporary tables. On Multiplex Systems logical server policy option TEMP_DATA_IN_SHARED_TEMP governs temporary tables creation in IQ_SHARED_TEMP,  default value is ‘off’. Changing this option

requires IQ server to be restarted. ‘with stop server’ clause can be used to automatically shutdown the affected nodes in logical server for

whom ls policy option is changed.

 

  •   If ‘TEMP_DATA_IN_SHARED_TEMP’ login policy server option is set to ‘off’, temporary objects are created in IQ_SYSTEM_TEMP.
  •   If ‘TEMP_DATA_IN_SHARED_TEMP’ login policy server option is set to ‘on’, temporary objects are created in IQ_SHARED_TEMP.


          Example:


                         - alter ls policy lp1 TEMP_DATA_IN_SHARED_TEMP=on with stop server

                         - alter logical server ls1 policy lp1 with stop server

                         - drop logical server ls1 with stop server


sp_iqsharedtempdistrib:


           This system stored  procedure reports the current shared temp space usage distribution. When sp_iqsharedtempdistrib is executed from

           Coordinator, it reports shared temp space distribution for all nodes. When sp_iqsharedtempdistrib is executed from Secondary nodes,

           it reports shared temp space usage for only that node.  Shared temporary space is reserved on each node in the multiplex on demand by

           the  Coordinator. Shared temporary space is reserved for a node in an allocation unit. Nodes can request and hold multiple allocation units

          based on their current demand. These allocation units are leased by the nodes to use more space as needed and return the space to a global

          pool when they don’t need it. Allocation units expire when the shared  temporary space usage decreases, or lease expires or when nodes

          are shutdown.


sp_iqstatus and sp_iqspaceused:


       When these system stored procedures are executed on Coordinator node, they report total space in use on IQ_SHARED_TEMP. When
      executed on Secondary nodes, these system stored procedures do not report space used by shared dbspace ie. IQ_SYSTEM_MAIN,

      IQ_SHARED_TEMP and user dbspaces. sp_iqspaceused takes following arguments:


         sp_iqspaceused(out mainKB            unsigned bigint,

          out mainKBUsed        unsigned bigint,

          out tempKB            unsigned bigint,

           out tempKBUsed        unsigned bigint,

           out shTempTotalKB     unsigned bigint,

           out shTempTotalKBUsed unsigned bigint,  

           out shTempLocalKB     unsigned bigint,  

           out shTempLocalKBUsed unsigned bigint,  

           out rlvLogKB          unsigned bigint,   

           out rlvLogKBUsed      unsigned bigint)

 

User-defined stored procedure myspace can be created that declares all seven sp_iqspaceused output parameters and then call sp_iqspaceused.

For myspace stored procedure code see:

         

            http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc38151.1602/doc/html/san1278453328180.html

 

sp_iqdbspace and sp_iqfile:


   sp_iqdbspace and sp_iqfile system stored procedure report include information for the IQ_SHARED_TEMP.


sp_iqcheckdb

 

     sp_iqcheckdb system stored procedure reports allocation information for IQ_SHARED_TEMP


                     sp_iqcheckdb ‘allocation dbspaceIQ_SHARED_TEMP’


      When executed from Coordinator Node, displays global block usage:


                 -  Blocks Total

                  -  Blocks Used

 

      When executed from any secondary node, sp_iqcheckdb reports blocks reserved for that secondary node:


                - Blocks Reserved for this Server Total


Exporting Data from Microsoft Access 2010

$
0
0
  1. Launch Microsot Access and Open Database.

 

access1.png

 

2. Highlight table accesstab right click -.Export ->ODBC Database -> Machine Data Source ->Sybase IQ Datasource -> OK

 

access2.png

 

3.  create table accesstab (col1 long integer, col2 text (255) in Microsoft Access and insert row

 

access3.png

 

4.  Highlight table accesstab right click -.Export ->ODBC Database -> Machine Data Source ->Sybase IQ Datasource -> OK

access7.png

 

access4.png

 

 

access5.png

 

5. Verify export by login into SAP IQ database.

 

acesss6.png

Using timeout server option to disconnect idle connections in SAP Sybase IQ

$
0
0

In SAP Sybase IQ, there is one server option '-ti' to be used to disconnect idle connections serverwide,

you can also use 'IDLE' connection parameter for individual connection when using dbisql or InterActive SQL utility to connect to SAP Sybase IQ server !

 

It is useful for ordinary client connections, such as dbisql, InterActive SQL or client wrote Java application !

It is also useful for web based browser to control number of user connections in a limited time range in case end user holding session for long time and no activity in that session !

 

However, for batching or reporting tasks, client can use the shared memory communications link to avoid timeout and being disconnected from IQ server !!

 

Please check Sybase IQ 1540 utility guide as following link for details on -ti option

http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc00168.1540/doc/html/san1276279392032.html

 

 

I have done a simple test to demostrate how to use -ti server option in Sybase IQ to timeout and disconnect idle connections.

 

Please take reference as following example:

 

Testing Steps:

 

 

A. Start IQ server with '-ti 10' ( connection idle 10 minutes will disconnect connections!!!)

 

 

B. Using following 3 way to connect to IQ server :

 

 

 

(1) connect from server host using Default method (shared memory) :

 

 

 

/* dbisql connection from host without CommLinks , default using shared memory , confirmed it will not timeout if idle 10 minutes */

 

h:w> dbisql -nogui -c "uid=DBA;pwd=sql;dbn=iqdemo"

 

(DBA)> select @@spid;

 

@@spid     

-----------

 

1         

 

(1 rows)

Execution time: 0.009 seconds

 

 

 

(2) connect from server host using TCPIP method :

 

 

 

/*  dbisql connection from host with CommLinks=tcpip , confirmed it will timeout if idle 10 minutes */

 

h:w> dbisql -nogui -c "uid=DBA;pwd=sql;eng=sgsinvm0257_iqdemo;CommLinks=tcpip"

 

(DBA)> select @@spid;

 

@@spid     

-----------

 

2         

 

(1 rows)

 

Execution time: 0.052 seconds

 

 

(DBA)> select @@spid;

 

Reconnected to database.

 

@@spid     

-----------

 

5         

 

(1 rows)

Execution time: 0.016 seconds

 

 

(DBA)>

 

 

(3) connect from client host using Interactive SQL with TCPIP method :

 

 

/* example : this  Connection handle 3 is client from notebook and you can see it used ‘tcpip’ commlink , that is ‘-ti’ will used for !!!! */

 

 

Please check attached screenshot for details !

timeout 3_1.jpg

 

timeout 3_2.jpg

 

 

C.  After 10 minutes and without any activity, check those 3 connection status :

 

(1) connect from server host using Default method (shared memory) :

Result: connHandle: 1 status is still alive !

 

 

 

(2) connect from server host using TCPIP method :

Result: connHandle: 2 status istimeout

 

 

 

(3) connect from client host using Interactive SQL with TCPIP method :

 

Result: connHandle: 3 status is timeout

 

 

 

/* grep from .iqmsg to show connect and disconnect log information !!! */

 

h:w> egrep -i '08/16 04' iqdemo.iqmsg |grep 'onnect' |more

 

 

  1. I. 08/16 04:00:00. 0000000653 Connect:  SA connHandle: 1000000650  SA connID: 28  IQ connID: 0000000653  User: DBA
  2. I. 08/16 04:00:00. 0000000653 Disconnect:  SA connHandle: 1000000650  SA connID: 28  IQ connID: 0000000653  User: DBA
  3. I. 08/16 04:00:03. 0000000299 Disconnect:  SA connHandle: 3  SA connID: 21  IQ connID: 0000000299  User: DBA
  4. I. 08/16 04:00:52. 0000000212 Disconnect:  SAconnHandle: 2  SA connID: 25  IQ connID: 0000000212  User: DBA
  5. I. 08/16 04:05:00. 0000000804 Connect:  SA connHandle: 1000000801  SA connID: 25  IQ connID: 0000000804  User: DBA
  6. I. 08/16 04:05:00. 0000000804 Disconnect:  SA connHandle: 1000000801  SA connID: 25  IQ connID: 0000000804  User: DBA
  7. I. 08/16 04:10:00. 0000000957 Connect:  SA connHandle: 1000000954  SA connID: 23  IQ connID: 0000000957  User: DBA
  8. I. 08/16 04:10:00. 0000000957 Disconnect:  SA connHandle: 1000000954  SA connID: 23  IQ connID: 0000000957  User: DBA
  9. I. 08/16 04:15:00. 0000001109 Connect:  SA connHandle: 1000001106  SA connID: 20  IQ connID: 0000001109  User: DBA
  10. I. 08/16 04:15:00. 0000001109 Disconnect:  SA connHandle: 1000001106  SA connID: 20  IQ connID: 0000001109  User: DBA
  11. I. 08/16 04:20:00. 0000001260 Connect:  SA connHandle: 1000001257  SA connID: 16  IQ connID: 0000001260  User: DBA
  12. I. 08/16 04:20:00. 0000001260 Disconnect:  SA connHandle: 1000001257  SA connID: 16  IQ connID: 0000001260  User: DBA
  13. I. 08/16 04:25:00. 0000001411 Connect:  SA connHandle: 1000001408  SA connID: 11  IQ connID: 0000001411  User: DBA
  14. I. 08/16 04:25:00. 0000001411 Disconnect:  SA connHandle: 1000001408  SA connID: 11  IQ connID: 0000001411  User: DBA
  15. I. 08/16 04:29:16. 0000001541 Connect:  SA connHandle: 4  SA connID: 18  IQ connID: 0000001541  User: DBA
  16. I. 08/16 04:30:00. 0000001564 Connect:  SA connHandle: 1000001560  SA connID: 9  IQ connID: 0000001564  User: DBA
  17. I. 08/16 04:30:00. 0000001564 Disconnect:  SA connHandle: 1000001560  SA connID: 9  IQ connID: 0000001564  User: DBA
  18. I. 08/16 04:30:29.0000001580 Connect:  SA connHandle: 5  SA connID: 27  IQ connID: 0000001580  User: DBA

Steps to configure RLV in an IQ 16 server

$
0
0

Purpose

The purpose of this document is to provide you with the steps to configure RLV on IQ 16 simplex server along with a simple example to see how the RLV feature works


  

Step 1: Configure RLV Memory

 

  • Server startup:

  -iqrlvmem <max MB>

  •   Runtime:

  sa_server_option ‘rlv_memory_mb’,<max MB>

 

Step 2: Create RLV Dbspace

 

create dbspace rlvdbsp using file rlvfile1 ‘rlvfile1.iq’ size 1000 IQ RLV STORE

 

Step 3: Create/enable RLV table

 

create table <table-name> (<column definition>) {ENABLE|DISABLE} RLV STORE

e.g

create table rlvtab(col1 int, col2 char(10) ENABLE RLV STORE

 

To enable/disable RLV storage for a table

 

alter table <table-name> {ENABLE|DISABLE| RLV STORE}

e.g

alter table rlvtab ENABLE RLV STORE

alter table rlvtab DISABLE RLV STORE

 

Note: option BASE_TABLES_IN_RLV_STORE registers newly created tables in RLV store after this option is set to ON.

 

Step 4: Control Versioning

 

Snapshot_Versioning: Controlling Row-level or Table-level versioning on a connection by connection basis

 

Syntax:

Set [Temporary] Option Snapshot_Versioning =‘Table-level’|’Row-level’

- default is ‘Table-Level’

- DBA can restrict Snapshot_Versioning values

Set Option Allow_Snapshot_Versioning = ‘Any’ | ‘Table-Level’ | ‘Row-level’

 

 

Example of setting up an RLV table and seeing how it works:

 

  1. C:\dbs\16rlv>c:\programdata\sybaseiq\demo\mkiqdemo.bat

This script creates the iqdemo database in the current directory: C:\dbs\16rlv

Continue <Y/N>? y

Setting up temporary files ...

 

Starting Server ...

 

Connecting to server via TCPIP ...

Using: uid=DBA;pwd=sql;eng=bld_demo_9577;dbn=utility_db;links=tcpip(host=vm-test;port=9577)

Creation completed.

 

Shutting down server ...

SQL Anywhere Stop

Server Utility Version 16.0.0.481

 

Volume in drive C has no label.

Volume Serial Number is B085-C0E8

 

Directory of C:\dbs\16rlv

09/24/2013  01:37 PM               662 iqdemo.cfg

09/24/2013  01:37 PM         3,874,816 iqdemo.db

09/24/2013  01:37 PM      104,857,600 iqdemo.iq

09/24/2013  01:37 PM            40,674 iqdemo.iqmsg

09/24/2013  01:37 PM        26,214,400 iqdemo.iqtmp

09/24/2013  01:36 PM             1,157 iqdemo.lmp

09/24/2013  01:37 PM           327,680 iqdemo.log

09/24/2013  01:37 PM      104,857,600 iqdemo_main.iq

8 File(s)    240,174,589 bytes

 

Demo database created successfully

 

2. Edit cfg file and add -iqrlvmem  2048

 

3. Start server

 

4. You can create rlv dbspace and table using SCC (SAP Control Center), but here is how to do it using dbisql. Run dbisql and execute

CREATE DBSPACE "rlvdbspace" USING  FILE "rlvdbfile1" 'c:\dbs\16rlv\rlvdbfile1.iq' SIZE 1024 MB RESERVE 1 MB IQ RLV STORE;

 

5. Set some options that impact RLV Merge

 

SET OPTION PUBLIC.RV_AUTO_MERGE_EVAL_INTERVAL='1';

SET OPTION PUBLIC.RV_MERGE_TABLE_NUMROWS=10000;

SET OPTION PUBLIC.BASE_TABLES_IN_RLV_STORE='ON'

SET OPTION PUBLIC.Snapshot_Versioning = 'Row-level';

 

Note: RV_AUTO_MERGE_EVAL_INTERVAL & RV_MERGE_TABLE_NUMROWS are set at a low value to be able to see RLV feature working in a small test environment.

 

6. Create example table and stored procedure to load the table

 

create table rlvtest

(

col1                        varchar(125) not null,

col2                        int not null,

col3                        varchar(1024),

create_time       datetime default getdate(),

create_by           varchar(20) default CURRENT USER,

mod_time           datetime default getdate(),

modified_by      varchar(20) default CURRENT USER,

primary key(col2)

)

ENABLE RLV STORE;

 

create procedure
p_load_rlvtest()

as

begin

  declare @li_line_num    int

  select @li_line_num = 1

  while (@li_line_num < 60000)

  begin
    insert into rlvtest (col1, col2, col3) values('Test data ' + convert(char(10), @li_line_num), @li_line_num,
    'The information contained in this email is confidential and intended only for the addressee.

    If you are not the intended recipient (or have received this e-mail in error), please notify the sender immediately and destroy this e-mail. Any
    unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden' )
    select @li_line_num= @li_line_num + 1
    if ((@li_line_num % 10000) = 0)

    begin
      commit
    end

  end

  select @li_line_num

end;

 

7. Run dbisql session and execute the following

call p_load_rlvtest()

 

8. Command to check the rlv merge history

select * from sysiqrlvmergehistory

 

rlv_mergehistory.png

 

 

9. What to look for in iqmsg

‘Current active row count’ will display how many rows have been inserted.

‘Mrg B’ – indicates the begin of the merge process

‘Mrg E’ – indicates the end of the merge process

You will see such entries in the iqmsg, but since I set merge_rows to 10000, the merge happens and row count is back to 0

 

I. 09/30 11:15:48. 0000000021 0000005656 Current active row count for table rlvtest is 10000

I. 09/30 11:15:48. 0000000021 0000005656 Current active row count for table rlvtest is 0

I. 09/30 11:15:48. 0000000021 Cmt 8120

I. 09/30 11:15:48. 0000000021 PostCmt 0

I. 09/30 11:15:48. 0000000021 Txn 8121 0 8121

I. 09/30 11:15:48. 0000000021 Cmt 8122

I. 09/30 11:15:48. 0000000021 PostCmt 0

I. 09/30 11:15:49. 0000000194 Txn 8123 0 8123

I. 09/30 11:15:49. 0000000194 Cmt 8124

I. 09/30 11:15:49. 0000000194 PostCmt 0

I. 09/30 11:15:49. 0000000195 Txn 8125 0 8125

I. 09/30 11:15:49. 0000000195 Cmt 8126

I. 09/30 11:15:49. 0000000195 PostCmt 0

I. 09/30 11:15:49. 0000000195 Txn 8127 0 8127

I. 09/30 11:15:49. 0000000195 Mrg B 788 NO

I. 09/30 11:15:49. 0000000195 Mrg D

I. 09/30 11:15:49. 0000000195 [20917]: Delete of 0 rows started for table:

I. 09/30 11:15:49. 0000000195 [20919]: Delete of 0 rows completed for table: rlvtest, 0 seconds.

I. 09/30 11:15:49. 0000000195 Mrg U

I. 09/30 11:15:49. 0000000195 Mrg I

I. 09/30 11:15:49. 0000000195 [20896]: Insert for 'rlvtest' completed in 0 seconds.  20000 rows inserted.

I. 09/30 11:15:50. 0000000195 Mrg E

I. 09/30 11:15:51. 0000000195 Cmt 8128

I. 09/30 11:15:51. 0000000195 PostCmt 0

 

 

Related Content
Recover RLV from failed recovery - http://scn.sap.com/docs/DOC-44821

RLV Memory Management and Monitoring - http://scn.sap.com/docs/DOC-50252

Resolving RLV Store Out of Memory Error - http://scn.sap.com/docs/DOC-45064

SAP IQ 16: Script to Generate Rebuild Indexes Scripts

$
0
0

After migrating a simplex SAP IQ 15.2 ESD#2 database to SAP IQ 16.0 SP3 running on Sun Solaris 10 over a Sparc 64 machine, I encounter that some queries where returning zero rows or returning erroneous data;  Also, some of the queries where making the database crash.  To solve those problems I rebuild the indexes in the table.

 

After installing and configuring SAP IQ 16.0 SP3 I upgrade the database using follow statement:


ALTER DATABASE UPGRADE SYSTEM PROCEDURE AS DEFINER OFF

 

And, set the following options:


SET OPTION PUBLIC.FP_NBIT_IQ15_Compatibility = OFF                                                                                                    

SET OPTION PUBLIC.CREATE_HG_WITH_EXACT_DISTINCTS = OFF

SET OPTION PUBLIC.REVERT_TO_V15_OPTIMIZER=OFF

 

Then, I performed consistency and allocation test


sp_iqcheckdb 'verify database'

sp_iqcheckdb 'allocation database'

 

Both DBCC retuned “No Errors Detected”.

 

And, I rebuild all the indexes of all the columns over 250 char long.

 

Then, while testing I encounter the errors described in the first paragraph.

 

So, to generate the statements to rebuild all the indexes on the table bellowing to the user DBA I wrote the following 2 scripts:

 

select 'sp_iqrebuildindex ''DBA.' + T.[table_name] + ''', ''index ' + I1.index_name + ' retier ''' + char(13) + 'go'

from sysidx I1, sysiqidx I2, systab T

where T.table_id = I1.table_id

and (I2.table_id=I1.table_id and I2.index_id = I1.index_id)

and T.creator = 1

and I2.index_type = 'HG'

 

select 'sp_iqrebuildindex ''DBA.' + T.[table_name] + ''', ''index ' + I1.index_name + '''' + char(13) + 'go'

from sysidx I1, sysiqidx I2, systab T

where T.table_id = I1.table_id

and (I2.table_id=I1.table_id and I2.index_id = I1.index_id)

and T.creator = 1

and I2.index_type <> 'HG'

 

And then, I just executed the generated scripts to rebuild all the indexes.

 

You can also modify these scripts to generate the statements to rebuild the indexes of any given table:


select 'sp_iqrebuildindex ''DBA.' + T.[table_name] + ''', ''index ' + I1.index_name + ' retier ''' + char(13) + 'go'

from sysidx I1, sysiqidx I2, systab T

where T.table_id = I1.table_id

and (I2.table_id=I1.table_id and I2.index_id = I1.index_id)

and T.[table_name] = 'myTable'

and T.creator = 1

and I2.index_type = 'HG'

Union

select 'sp_iqrebuildindex ''DBA.' + T.[table_name] + ''', ''index ' + I1.index_name + '''' + char(13) + 'go'

from sysidx I1, sysiqidx I2, systab T

where T.table_id = I1.table_id

and (I2.table_id=I1.table_id and I2.index_id = I1.index_id)

and T.[table_name] = 'myTable'

and T.creator = 1

and I2.index_type <> 'HG'

 

Or, you can use these scripts to build dynamic SQL that will rebuild all the indexes in the database.

 

This post has first been published on my personal blog at http://wp.me/p1Gouv-6N

ASE 15.7: Create a Remote Server to SAP IQ 16.0

$
0
0

1. Create the remote server entry in the interfaces file;  sql.ini (Windows) or interfaces (UNIX).

 

[myIQServer]

master=TCP,myIQHost,4091

query=TCP,myIQHost,4091

 

2. Add the remote server

 

sp_addserver 'myIQServer', ASIQ, myIQHost:4091'

 

3. Add the external login (SAP IQ) to be used by the local user (ASE) when connecting the SAP IQ database

 

sp_addexternlogin myIQServer, myLocalUser, myIQUser, myIQPassword

 

4. Verify connectivity using passthrough

 

connect to myIQServer

 

select @@version

 

@@version

------------------------------------------------------------------------------

Sybase IQ/16.0.0.653/131122/P/sp03/Sun_Sparc/OS 5.10/64bit/2013-11-22 01:16:26

 

1 record(s) selected [Fetch MetaData: 0ms] [Fetch Data: 0ms]

 

[Executed: 12/03/2014 2:00:20 PM] [Execution: 2ms]

 

 

disconnect from myIQServer



This document has previously been published in my personal blog at ASE 15.7: Create a Remote Server to SAP IQ 16.0 | ARBIME


SAP IQ: How to Kill Runaway Queries

$
0
0

In environments where lot of researches query the database, it is necessary to allow the users to run queries that are resource and time intensive.  Even if there are database options that automatically avoid runaway queries, we cannot be too restrictive in this kind of environments. This flexibility allows users to execute runway queries (queries that never end or that has execution time considerably longer than expected and, consume system resources). This document explains how to identify and kill a runway query.

 

  1. Log into the database using an account with the drop connection system privilege (IQ 16) or DBA or operator authority (IQ <16).
  2. Execute sp_iqcontext to identify the connection with the runaway query, right down the ConnHandle if the connection, you will need it to drop the connection.  The CmdLine field of the result contain the text of the runaway query.
  3. Optionally, execute sp_iqtransaction to see if the transaction is still active.
  4. Execute ‘drop connection <ConnHandler>’ to drop the connection and kill the transaction.

 

This document were fist posted on my personal blog at SAP IQ: How to Kill Runaway Queries | ARBIME

Exporting Data from SAP HANA into SAP IQ

$
0
0

Steps listed below can be used to export data from SAP HANA into SAP IQ.

 

1. Set environment variables

 

        Edit $SYBASE/IQ-16_0/IQ-16_0.csh and  set HANA_HOME, ODBCHOME and ODBCINI environment variables.

 

                  setenv HANA_HOME /hana/sap/hdbclient/libodbcHDB.so

                  setenv ODBCHOME /hana/rel16_iq_sp03/

                  setenv ODBCINI /hana/rel16_iq_sp03/odbc.ini

 

2. Add HANA and IQ entries in odbc.ini

 

[iq1603]
DRIVER=/hana/rel16_iq_sp03/IQ-16_0/lib64/libdbodbc16.so
EngineName=iq1603
CommLinks=tcpip(host=myhost;port=9286)
Userid=DBA
Password=sql
AutoStop=no
DatabaseName=iqdb

[SB1]
Driver=/hana/sap/hdbclient//libodbcHDB.so
ServerNode=myhost:30315

 


3. Source $SYBASE/IQ-16_0/IQ-16_0.csh  and restart IQ server. IQ server has to be started with HANA_HOME and OBCINI environment

variables.

 

 

4. Create HANA remote server named SB1 on the machine "myhost"  with listening on port 30315:

 

dbisql -c 'dsn=iq1603' -nogui

(DBA)> create server SB1 class 'hanaodbc' using 'dsn=SB1'
Execution time: 0.067 seconds

 

5. Create externlogin to map the local user named DBA to the user SYSTEM with the password 'Saphanasql1'

 

DBA)> create externlogin DBA to SB1 remote login SYSTEM identified by 'Saphanasql1'

Execution time: 2.111 seconds

 

6. Confirm connection to HANA instance SB1

 

(DBA)>sp_remote_tables SB1

(NULL)
                                                 _SYS_STATISTICS

                  HOST_ONE_DAY_FILE_COUNT

 

 

(NULL)
                                                 _SYS_STATISTICS

                  HOST_RECORD_LOCKS

(NULL)
                                                 _SYS_STATISTICS

                  STATISTICS_LAST_CHECKS

 

 


(1090 rows)

              

 

7. Create schema ,table on  HANA instance SB1

 

(DBA)> forward to SB1
0 row(s) affected
Execution time: 0 seconds

(DBA)> create schema hanaiq
0 row(s) affected
Execution time: 0.23 seconds

(DBA)> create table hanaiq.t1(col1 int)
0 row(s) affected
Execution time: 1.321 seconds

 

8. insert row in to HANA table hanaiq.t1

 

(DBA)> insert into hanaiq.t1 values(1)

0 row(s) inserted

Execution time: 0.07 seconds

(DBA)> commit

Execution time: 0.031 seconds

(DBA)> select * from hanaiq.t1
       COL1
-----------
          1

(1 rows)
Execution time: 0.037 seconds

(DBA)> forward to
Execution time: 0.001 seconds

 

9. create a proxy table named TAB1 for the remote table hanaiq.t1 at the remote server SB1

 

(DBA)> create existing table DBA.TAB1 (COL1 int) at 'SB1.HANAIQ..T1'
Execution time: 0.441 seconds

(DBA)> select count(*) from DBA.TAB1
    count()
-----------
          1

(1 rows)

Execution time: 0.063 seconds

(DBA)>

 

 

 

 

          

             


[SAP IQ] How to unlock the users.

$
0
0

There are several reasons to be locked.

 

1.  A user can be locked because user belongs to a login policy with locked=ON.(locked)  
2.  A user can also be locked if user exceeds the maximum number of failed login attempts allowed by the user's login policy.(max_failed_login_attempts)
    You can verify the reason why user is locked with the following function.
3. A user can also be locked if user exceeds the maximum number of days that can elapse between two successive logins by the same user.(max_days_since_login)

 

The manner in which you unlock an account depends on how it was originally locked.

1.   If a user account is locked because it is assigned to a login policy with the locked option set to ON, reassign the user to a login policy with the locked option set to OFF.
      Resolution =>  ALTER LOGIN POLICY TEST_LP locked=OFF;
      
2.   If a user account is locked because it has exceeded the MAX_FAILED_LOGIN_ATTEMPTS or MAX_DAYS_SINCE_LOGIN, issue the ALTER USER statement with the RESET LOGIN POLICY option.
     Ex) sa_get_user_status;
      Resolution =>  ALTER USER gjang RESET LOGIN POLICY;


user_id


user_name


connections


failed_logins


last_login_time


locked


reason_locked


101


gjang


0


4


2013-10-08 10:55:00.0


1


maximum failed logins exceeded

 

See Also

 

1. For details for unlocking users
    http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc00170.1540/doc/html/san1288042729924.html

[SAP IQ] Internal Options to enhance parallelism.

$
0
0

There are some internal options to enhance parallelism.

 

1) DML_OPTIONS56=1;
- run distinct operations in parallel.

 

2) DML_OPTIONS51/52
- help increase parallelism and help with sorts.
   set the dataflow work unit sizes.
   The lower the value the higher the parallelism.

 

3) DML_Options8=65536;  
- parallelize the radix work.

   You can allow this query to be parallelized(insert-select).

Login Redirection

$
0
0

Login Redirection feature in SAP IQ 16 allows load balancing by redirecting login connecting to overloaded node to less node in the same logical server. Login Redirection can be controlled by using login server policy options:

 

                    - LOGIN_REDIRECTION

                    - REDIRECTION_WAITERS_THRESHOLD

 

For information related to these options see:

 

 

                 http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc01839.1602/doc/html/jan1292624444594.html

 

          

                 http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc01839.1602/doc/html/jan1328621506262.html

            

Steps to setup Login Redirection:


In my setup, I have iq1602c, iq1602q1, iq1602q2, and iq1602q1 and iq1602q2 are members of logical server ls1

 

[sybase@myhost]$ dbisql -c 'dsn= iq1602c' -nogui

 

(DBA)> CREATE LOGICAL SERVER ls1 MEMBERSHIP (iq1602q1,iq1602q2);

 

  Execution time: 0.062 seconds

  (DBA)> alter ls policy root TEMP_DATA_IN_SHARED_TEMP=ON login_redirection = On dqp_enabled = 1 redirection_waiters_threshold = 5

   Execution time: 0.106 seconds

 

[sybase@myhost]$ dbisql -c 'dsn= iq1602c' -nogui

  

(DBA)> alter login policy root add logical server ls1

  Execution time: 0.059 seconds

 

we have iq1602q1 and iq1602q2 in  logical server ls1.  Now if we connect to Coordinator iq1602c,  login will be redirected to either iq1602q1
or iq1602q2.

  

dbisql -c 'uid=DBA;pwd=sql;eng= iq1602c;LS=ls1;NODETYPE=WRITER' -nogui

  

(DBA)> select @@servername

 

@@servername

 

    --------------------------------------------------------------------------------

 

iq1602q1

 

(1 rows)

   Execution time: 0.043 seconds

 

    Now for testing shutdown iq1602q1,  connection will be redirected to iq1602q2

  

dbisql -c 'uid=DBA;pwd=sql;eng= iq1602c;LS=ls1;NODETYPE=WRITER' -nogui

  

(DBA)> select @@servername

 

@@servername

     --------------------------------------------------------------------------------

 

iq1602q2


if connection redirection is enabled and login/connection  will redirect connections when:

 

  • ŸIf the connection is being made to a server which is not member of thespecified logical server
  • ŸIf there are more waiters at the gate than the specified threshold of waiters in the -iqgovern queue.
  • ŸIf the specified node type does have role different than that requested.

 

 

 

 

 

http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc01839.1602/doc/html/jan1328621506262.html

RLV Memory Management and Monitoring

$
0
0


In Memory row-level feature is available in SAP IQ 16.  This feature allows more than one user to update same table concurrenlty as long as they are adding or modifying different rows. You will need to configure IQ server with additional memory for RLV store ie. in addition to main cache, temp cache adn size of large memory pool, you will need need additional memory for RLV store.

 

RLV memory can be configured wtih startup switch -iqrlvmem (default 2G).

 

# iqdemo.cfg

# ------------------------------------------------------------

# Default startup parameters for the ASIQ demo database

# ------------------------------------------------------------

-n  iq1602

-x  tcpip{port=9447}

# The following parameters are also found in the configuration file

# /usr/sybase/rel16_iq_sp02/IQ-16_0/scripts/default.cfg.  Any parameters not specified below

# and not in the start up parameter list, will be added by start_iq

# using default.cfg as a guide.

-c  512m

-gc 20

-gd all

-gl all

-gm 10

-gp 4096

-iqmc 2000

-iqtc 2000

-zr sql

-iqrlvmem 4096

 

 

sa_server_option SA system procedure can be used to increase RLV memory dynamically:

 

   sa_server_option 'RLV_memory_mb', 8192

 

  This will configure RLV store size to 8192MB

 

  RLV memory usage depends on several factors:

 

  •   Table Schema(number of fixed and variable length columns and width of the columns)
  •   Concurrent loads and amount of data being loaded
  •    Number of cores
  •    Merge frequency

 

Every single RLV enabled table will require an in-memory table to hold its data. The memory requirement of this RLV store will change during the life of the table while it is RLV enabled. The initial size of RLV store needed during first DML can be calculated approximately using following formula:

 

 

 

#COLS = #columns + 1 (pseudo column)

Base RLVstore (BRS) = ( Initial setup of the RLV store )

                BRS ~= (#COLS)  x 16MB

 

Fixed datatypes of a subfragment uses default 16MB to store data in the RLV store. Variable datatypes uses default 512KB and bit datatypes uses 128KB default size.

 

    Example:

             Number of Cores=4

             Number of Columns=303

             RLV memory can be approximately calculated using:

                  16mb*4*303=~19GN

             for Singleton insert using only 1 core,  RLV memory used:

                   16mb *1*303= ~4.8GB

 

     Data is store in table fragment. Each table fragments is further subdivided into subfragments. Multiple table versions of the table fragments may be required to handle multiple transactions. Each table version will have the RLV memory size requirement of at least equal to this BRS(Base RLV Store). Multiple transactions will create multiple table store versions that need to be stored in memory simultaneously. Multiple table versions can quickly increase the memory requirement of the RLV enabled table. Old transactions can hold onto old references to the in-memory store whch will also consume memory.

 

  RLV Out Of Memory Errors:

  

Exception Thrown from rvlib/rv_TabStoren.cxx:178, Err#5, tid 3227 origtid 3227

O/S Err#: 0,ErrID: 15008 (rv_PersistenceException); SQLCode: -1013129, SQLState: 'QRLV1',Severity: 14

[22159]: RLVStore has run out of memory Exception Thrown from rvlib/rv_TabStoren.cxx:178, Err# 5, tid 3 origtid 3227

O/S Err#: 0, ErrID: 15008 (rv_PersistenceException); SQLCode: -1013129, SQLState: 'QRLV1', Severity: 14

 

   Explanation A:

              - To much data is being modified/ added by RLV transactions at once  

  Fix:
           -  Reduce the size of update/inserts and increase frequency of commits to  keep amount of data modified/transaction.

 

  Explanation B: 
          - Automated merge is not able to keep up with RLV workload

  Fix:  
           - Review RLV merger  history table(SYSIQRLVMERGEHISTORY)
    
               - Adjust the automated merger period(RV_AUTO_MERGE_EVAL_INTERVAL)

               -  Adjust automated thresholds, RV_MERGE_TABLE_NUMROWS,     RV_MERGE_TABLE_MEMPERCENT, and RV_MERGE_NODE_MEMSIZE  RLV Options.

 

   

   Explanation:

                   - Maximum RLV memory configure too low.

                   -increase maximum RLV memory using server startup switch –iqrlvmem

                  or

                   -increase maximum RLV memory using sa_server_option

    
                        sa_server_option  ‘rlv_memory_mb’, <max MB>

                  - issue manual merge to free of RLV memory using

      
                      sp_iqmergerlvstore (merge_type, table_name, [table_owner])

 

Note: if low or out of memory , then issue blocking merge, as non-blocking merge needs to setup additional  in-memory table fragments, which won’t be
possible,if already low/out of memory

 

Monitor:

  •     Use sp_iqrlvmemory system stored procedure to monitor RLV memory usage per table.

       

   (DBA)> sp_iqrlvmemory


    table_id  fragments      total       data dictionary     bitmap

    -----------------------------------------------------------------

      1089          1          6          0          3          2

       822          1         64         64          1          1


   (2 rows)

 

  •     Use sp_iqstatus system stored  procedure to monitor memory used by the RLV store

 

    (DBA)> select * from sp_iqstatus() where name like '%RLV%'

    Name


                Value

    ----------------------------------------------------------------


    RLV memory limit:


                4096Mb

    RLV memory used:

 

                76Mb

 

   (2 rows)

Setting Remote Procedure Call

$
0
0

This doc talk about how to set up RPC to other server, when RPC is  embedded with in begin … end block.   When Remote procedure is
created, engine has no knowledge of what Remote Procedure does and therefore It cannot implicitly define result set for Remote Procedure.

 

 

  If Remote Procedure is executed using  "call foo(...)" in DBISQL or dbisqlc, the application will open a cursor on the call statement
and the proper "cursor describe stuff" will automatically happen. BUT, if you attempt to execute a "call foo(...)" within a procedure
or batch, the engine does not automatically open a cursor on the call. Instead, it is up to the user to explicitly force a cursor to be opened either by
declaring a cursor within the stored procedure or batch or using the RPC in the from clause of a select statement. Furthermore, if the RPC is going

to be used in the from clause, then the RPC *MUST* be created with a RESULT clause in order to allow the engine to predefine the schema

of the expected result set.

 

 

STEPS:

 

Add source and target SAP Sybase IQ server entries in the interfaces. Add $ODBCINI and $ODBCHOME environment variables in $SYBASE/IQ-15_4/IQ-15_4.csh  and then source to setup environment variables

  1. Create remote server

  CREATE SERVER "iqdemo1543" CLASS 'SAODBC' USING 'dsn=iqdemo1543';

 

 

   2. Create  Externlogin

 

       CREATE EXTERNLOGIN "DBA" TO "iqdemo1543" REMOTE LOGIN "DBA" IDENTIFIED BY 'sql';

 

 

   3. Create Remote Procedure:

 

   create "DBA"."remwho"()

       result(

      ConnHandle unsigned bigint,

     IQconnID unsigned bigint,
 
     Userid varchar(255),

    ReqType varchar(255),

    IQCmdType char(32),

    BlockedOn unsigned bigint,

    BlockUserid varchar(255),

   IQCursors unsigned integer,

   IQThreads unsigned integer,

   IQIdle integer,

   SAIdle integer,

   TempTableSpaceKB unsigned bigint,

   TempWorkSpaceKB unsigned bigint )

   at 'iqdemo1543...sp_iqwho'

 

   4. Execute Remote Procedure

 


begin select * from remwho() end

ConnHandle           IQconnID             Userid
                                                        
ReqType

IQCmdType                        BlockedOn            BlockUserid



                                                                IQCursors

IQThreads  IQIdle      SAIdle      TempTableSpaceKB     TempWorkSpaceKB

--------------------------------------------------------------------------------

-----------------------------------------------------------------------

23                   5625                 DBA
                                                        
OPEN

IQUTILITYOPENCURSOR              0                    (NULL)
                                                             
0

1

      0           0           0                    0



(1 rows)


Execution time: 0.025 seconds

Creating ASEODBC Connection from SAP Sybase IQ to SAP Sybase IQ Adaptive Enterprise Server

$
0
0

Steps listed below can be used to setup remote connection from SAP Sybase IQ and SAP Sybase Adaptive Server Enterprise. First you will need to install  OCS 157 64 bit  on IQHOST in diferent location for using ASE odbc driver

 

 

 

1. setup Environment variables.

 

cd to $SYBASE/IQ-15_4 and edit IQ-15_4.csh and add:

 

   setenv LIBPATH "${LIBPATH}:/usr/sybase/rel157/DataAccess64/ODBC/lib"

   setenv PATH "${PATH}:/usr/sybase/rel157/DataAccess64/ODBC/lib"

 

setenv ODBCINI /usr/sybase/rel154_iq_esd3/odbc.ini

  setenv ODBCHOME /usr/sybase/rel154_iq_esd3/

 

   2. source IQ-15_4.csh

 

   3. Add SAP Adaptive Server Enterprise and SAP Sybase IQ Server entries in interfaces

 

   % cat $SYBASE/interfaces

        aixase157

          master tcp ether ASEHOST 8412

          query tcp ether ASEHOST 8412

 

   iq154

          master tcp ether IQHOST 9522

          query tcp ether IQHOST 9522

 

   4. % cat $SYBASE/odbc.ini

   [aixase157]

  Description = Sybase ODBC Data Source

  UserID = sa

  Password =

  Driver = /usr/sybase/rel157/DataAccess64/ODBC/lib/libsybdrvodb.so

  Server = ASEHOST

  Port =  8412

  Database = pubs2

  UseCursor = 1

 

   [iq154]

  DRIVER=/usr/sybase/rel154_iq_esd3/IQ-15_4/lib64/libdbodbc12.so

  EngineName=iq154

  CommLinks=tcpip(host=IQHOST;port=9522)

  Userid=DBA

  Password=sql

  AutoStop=no

  DatabaseName=iqdemo

 

5. stop and restart IQ server

  6. Create Server and externlogin

     % dbisql -c 'dsn=iq154' -nogui

   (DBA)> select @@version

        @@version

 

  --------------------------------------------------------------------------------

Sybase IQ/15.4.0.3027/130814/P/ESD 3.6/Enterprise Linux64 - x86_64 - 2.6.18-194.el5/64bit/2013-08-14 03:34:24

(1 rows)

 

   Execution time: 0.014 seconds

 

  (DBA)> create server aixase157 class 'ASEODBC' usin

'Description=Sybase ODBC Data Source;Driver=/indigo61/sbagai/rel157/DataAccess64/ODBC/lib/libsybdrvodb.so;

UserID=sa,Password=;Server=indigo6;Port=8412;Database=pubs2;UseCursor=1'
;

  Execution time: 0.036 seconds

 

(DBA)> create externlogin DBA to aixase157
remote login 'sa' ;

 

Execution time: 0.032 seconds

 

(DBA)> forward to aixase157

 

0 row(s) affected

Execution time: 0.01 seconds

 

(DBA)> select @@version

 

expr_1

  --------------------------------------------------------------------------------

---------------

 

Adaptive Server Enterprise/15.7.0/EBF 19803SMP ESD#01 /P/RS6000/AIX 6.1/aseasap

 

/2918/64-bit/FBO/Wed Feb  8 06:09:12 2012

 

 

(1 rows)

 

Execution time: 0.85 seconds

 

(DBA)>

Forced Recovery Steps for Coordinator

$
0
0

if you are unable to start SAP IQ  Coordinator Server or  Database  in multiplex system,  check .iqmsg and .srvlog for diagnostics information, depending

on the error you might be able to recover Coordinator using  forced recovery steps. Most of the time assertion failed  error messages are SA errors. Forced

recovery will not work for SA assertion failed  error messages, you will need to restore from backups. It is advisable to contact  SAP Sybase

technical support for assistance. For IQ  issues (s_buf, freelist),  you can follow steps listed below for performing  forced recovery. For any

questions or concerns contact SAP Sybase Technical support.

 

Before using forced recovery, it is very important to  restrict access to the database, you can either use -gm 1(allows single connection plus

one DBA connection so that DBA can connect and drop others in case of emergency, or  issue sa_server_option('disable_connections', 'on')

after starting server in forced recovery mode, after forced recovery procedures, issue sa_server_option('disable_connections', 'off') on the same

connection.

 

Steps:

 

1. Shut down all  secondary nodes using stop_iq or stop_server.sh

2. Start Coordinator with -iqfrec , -iqmpx_sn 1 , -gn 1  and -gd 1

 

       start_iq -n <coordinator_server>  @params.cfg  -gd dba -gm 1 -iqfrec <coord db name without .db extn> -iqmpx_sn 1  <coord db name with .db extn>

 

      start_iq  -n < coordinator_server>  @params.cfg  -gd dba  -gm 1 -iqfrec  coord -iqmpx_sn 1 coord.db

 

3. Forced recovery marks all allocation within database as in use, run sp_iqcheckdb in  dropleaks mode to reset allocation map to correct

state.

            set temporary option dbcc_log_progress='on' ;

            sp_iqcheckdb 'dropleaks database';

 

    If there are no errors and sp_checkdb displays message 'Freelist Updated'  you have recovered leaked blocks.  Continue to Step 4

 

   if incosistency is found, follow the instructions for dropping inconsistent indexs/tables or columns to drop inconsistent objects in Administration:

   Backup, Restore and Data Recovery (Chapter: System Recovery and Database Repair)

 

             http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01759.1601/doc/pdf/iqbackup.pdf

 

4.  Issue Checkpoint

5. Shutdown Coordinator using stop_iq or stop_server.sh script

6.  Restart Coordinator normally without -iqfrec, -iqmpx_sn 1 , -gd 1 and -gm 1 switches

 

          ./start_server.sh

 

      Note:  start_server.sh and stop_server.sh scripts are generated by Sybase Control Center in  Server .db directory.

 

7.  If corruption was in IQ_SYSTEM_MAIN then run_sp_iqcheckdb on IQ_SYSTEM_MAIN else run dbcc on entire database.

 

           set temporary option dbcc_log_progress='on' ;

           sp_iqcheckdb 'verify dbspace IQ_SYSTEM_MAIN';

           sp_iqcheckdb 'verify database'

 

8.  Synchronize Secondary Nodes.


Query Plan Changes- IQ 16

$
0
0

Query html plans are generated for load execution plans.  Load execution html plans are generated for these statements:

 

  • Alter table add column (with default value)
  • Create Index
  • Declare cursor for update
  • Insert..into
  • Insert..location
  • Insert..values
  • Load..into
  • Parallel IQ statements

 

To Generate html plans for load execution, you will need to set appropriate Query options:

 

  • set option public.query_plan='on'
  • set option public.query_detail='on'
  • set option public.query_plan_as_html='on'
  • set option public.query_plan_after_run='on'
  • set option public.query_timing='on'
  • set option public.query_plan_min_time='on'
  • set option public.query_plan_as_html_directory='on'

 

For detailed information on these options:

 

http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc00801.1600/doc/pdf/iqrefso.pdf

 

 

These option can be set as temporary option as well,  Html plans should be only used to evaluate the performance of a particular query or load. Query_plan database option shouldn't be set to ON as it can significantly impact performance, especially as the volume of insert..value statement increases.

 

Query Plan Nodes:

 

  • Root Node:  This node is created at very top of the Data Flow Tree.
  • Sequencer:  In the Data Flow Tree, this node has two or more childen.  Pass1 branch of load has Child1 and Pass2 branch of load  has Child2.
  • Parallel Combiner: This Node takes multiple streams of rows from below, and 1 stream of rows is produced.
  • Index Insert: Insert into indexes. Sorted data values used for HG or WD or TEXT are retrieved and inserted into index. Only unique constraint is applied during index insert.
  • RowID Generator: Unique rowid  for each row is generated.
  • Leaf:  This node is created at the bottom of the Data Flow Tree.

 

New Query Plan Nodes:

  • Distinct Sort(inserter):  Node in Data Flow Tree indicating early aggregation algorithm being used.
  • Distinct Sort(retriever): Node in Data Flow Tree indicating early aggregation algorithm being used.
  • Grouping Sort(inserter): Node in Data Flow Tree indicating early aggregaton algorithm being used.
  • Grouping Sort(retriever):  Node in Data Flow Tree indicating early aggregation algorithm being used.
  • Join(Asymmetric Sort-Merge): Node in Data Flow Tree indicating asymmetric sort-merge join being used.
  • Join(Asymmetric Sort-Merge Pushdown): Node in Data Flow Tree indicating asymmetric sort-merge join being used.
  • Order By(Per-Work-Unit):  Node in Data Flow Tree indicating an operator that performs a sort on a work-unit worth of data. For example, it displays on the left branch under the  Asymmetric Sort-merge Join node.

Global Transaction Resiliency

$
0
0

Global Transaction Resiliency is HA functiionality available in IQ 16. This functionality allows Global Transactions to survive temporary communcation and temporary Coordinator failures.

 

 

Global Transaction is started by DML( delete/insert/load/update) on Multiplex Secondary(Writer) Node.  Global Transactions are linked to INC( Internal Internode Communication) connection  between Secondary(Writer) node and Coordinator.  RW operation executed on Secondary(Writer) node behave as they were executed the Coordinator and does acquire table locks.  Global Transaction failure can occur due to Coordinator failure(crash due to Hardware/Software issues, Coordinator Shutdown), Failover and INC(internal internode communication) . In such scenarios active Global DML transactions on Secondary(Writer) node  will fail,  secondary node will not be able to commit/rollback these transactions, even after Coordianator is available after some time. These transactions will need to be applied again.

 

With Global Transaction Resiliency functionality, Global Transactions are suspended and resumed, when INC connection is restored in following conditions:

  • Coordinator Crash(Hardware and Software issue)  and restart.
  • Coordinator shutdown and restart
  • Coordinator failover to New Coordinator.
  • INC(internal internode communication) failure and re-establishment.

 

In case of temporary failure of INC/Coordianator Global DML transactions on Secondary(Writer) node are kept in suspended state, and failures are resolved within timeout set with MPX_LIVENESS_TIMEOUT database option,  then global transaction resumes, as if there were no failures. If the failures persists longer than the timeout set, then global transaction cann't be resumed, Secondary(Writer) rolls back suspended global transaction.

 

System Stored Procedure sp_iqconnection can be used to check connection status on Secondary(Writer) Node.

 

  • INCConnName:  The Name of the underlying INC connection for a user connection. The datatype for INCConnName is varchar(255). If sp_iqconnection shows an INC connection Name for a suspended user connection, that user connection has an associated INC connection that is also suspended.
  • INCConnSuspended: The value of "Y" in this field indicates that the underlying INC connection for a user connection is in suspended state. The "N" indicates that the connection is not suspended.

 

For complete details on sp_iqconnection:

 

 

http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc38151.1600/doc/pdf/iqrefbb.pdf

 

System Stored Procedures sp_iqmpsuspendedconninfo() can be used to for details about currently suspended connections and transactions on the Coordinator Node.

and

sp_iqmpxincstatistics()  can be used for snapshot  of the aggregate statistics of  the INC(internal Internode communication) status since server startup as if the moment of execution

 

For complete details:

 

http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01839.1600/doc/pdf/iqmpx.pdf

 

 

 

Database options to control Globaal Transaction Resilency behavior:

 

mpx_heartbeat_frequency:  Default:  60 seconds

                                             Minimum:  2 seconds

                                             Maximum: 3600 seconds

 

mpx_heartbeat_frequency option sets the interval at which Secondary node internally polls Coordiator availability.

 

mpx_liveness_timeout:  Default: 3600 seconds

                                       Minimum:  0 seconds

                                       Maximum: 3600 seconds

 

mpx_liveness_timeout  option sets the timeout in seconds before a Secondary(Writer) Node rolls back suspended global transactions, if Coordiator is not available for a timeout greater than this option values.  Cooridnator rollback suspended global transactions if Secondary(Writer) Node fails to resume the transactions for a period greater than twice of this option values.

 

For complete details:

 

http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01839.1600/doc/pdf/iqmpx.pdf

 

 

Global Transaction Resiliency functionality does not affect trasactions initiated on the Coordinator Node. DDL commands from Secondary Nodes(Writer/Reader) will fail if Coordiator Node is not available.

Featured Content for SAP IQ

$
0
0

Guinness World Record for largest data warehouse - 12.1 petabytes!

SAP IQ has done it again! With the help of our partners  BMMsoft, HP, Intel, NetApp, Red Hat, and MD&Profy set the Guinness World's record for the largest data warehouse at 12.1 petabytes (PB). It is an exciting milestone in SAP's Big Data strategy and proof o four leading technologies. View the press release and read the blog by Mike Hagman for details.

 

SAP IQ and the In-Memory Data Fabric
SAP In-Memory Data Fabric delivers real-time, in-context business decisions for everyone while simplifying the IT landscape. Allowing for complex analyses, plans and simulations to be done with real-time data and made available immediately.  SAP IQ provides petabyte scale expansion - never put data in cold storage again. SAP IQ provides:

  • Faster, easier access to petabytes of data at any level of detail
  • Simplifies administration and streamlined IQ Landscape
  • Dramatically improved performance

 

SAP IQ 16 News

Subscribe for the quarterly newsletter to hear the latest news on SAP IQ and SAP ASE!

http://pagead2.googlesyndication.com/simgad/15713009812666520838

 

 

Using timeout server option to disconnect idle connections in SAP Sybase IQ

$
0
0

In SAP Sybase IQ, there is one server option '-ti' to be used to disconnect idle connections serverwide,

you can also use 'IDLE' connection parameter for individual connection when using dbisql or InterActive SQL utility to connect to SAP Sybase IQ server !

 

It is useful for ordinary client connections, such as dbisql, InterActive SQL or client wrote Java application !

It is also useful for web based browser to control number of user connections in a limited time range in case end user holding session for long time and no activity in that session !

 

However, for batching or reporting tasks, client can use the shared memory communications link to avoid timeout and being disconnected from IQ server !!

 

Please check Sybase IQ 1540 utility guide as following link for details on -ti option

http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc00168.1540/doc/html/san1276279392032.html

 

 

I have done a simple test to demostrate how to use -ti server option in Sybase IQ to timeout and disconnect idle connections.

 

Please take reference as following example:

 

Testing Steps:

 

 

A. Start IQ server with '-ti 10' ( connection idle 10 minutes will disconnect connections!!!)

 

 

B. Using following 3 way to connect to IQ server :

 

 

 

(1) connect from server host using Default method (shared memory) :

 

 

 

/* dbisql connection from host without CommLinks , default using shared memory , confirmed it will not timeout if idle 10 minutes */

 

h:w> dbisql -nogui -c "uid=DBA;pwd=sql;dbn=iqdemo"

 

(DBA)> select @@spid;

 

@@spid     

-----------

 

1         

 

(1 rows)

Execution time: 0.009 seconds

 

 

 

(2) connect from server host using TCPIP method :

 

 

 

/*  dbisql connection from host with CommLinks=tcpip , confirmed it will timeout if idle 10 minutes */

 

h:w> dbisql -nogui -c "uid=DBA;pwd=sql;eng=sgsinvm0257_iqdemo;CommLinks=tcpip"

 

(DBA)> select @@spid;

 

@@spid     

-----------

 

2         

 

(1 rows)

 

Execution time: 0.052 seconds

 

 

(DBA)> select @@spid;

 

Reconnected to database.

 

@@spid     

-----------

 

5         

 

(1 rows)

Execution time: 0.016 seconds

 

 

(DBA)>

 

 

(3) connect from client host using Interactive SQL with TCPIP method :

 

 

/* example : this  Connection handle 3 is client from notebook and you can see it used ‘tcpip’ commlink , that is ‘-ti’ will used for !!!! */

 

 

Please check attached screenshot for details !

timeout 3_1.jpg

 

timeout 3_2.jpg

 

 

C.  After 10 minutes and without any activity, check those 3 connection status :

 

(1) connect from server host using Default method (shared memory) :

Result: connHandle: 1 status is still alive !

 

 

 

(2) connect from server host using TCPIP method :

Result: connHandle: 2 status istimeout

 

 

 

(3) connect from client host using Interactive SQL with TCPIP method :

 

Result: connHandle: 3 status is timeout

 

 

 

/* grep from .iqmsg to show connect and disconnect log information !!! */

 

h:w> egrep -i '08/16 04' iqdemo.iqmsg |grep 'onnect' |more

 

 

  1. I. 08/16 04:00:00. 0000000653 Connect:  SA connHandle: 1000000650  SA connID: 28  IQ connID: 0000000653  User: DBA
  2. I. 08/16 04:00:00. 0000000653 Disconnect:  SA connHandle: 1000000650  SA connID: 28  IQ connID: 0000000653  User: DBA
  3. I. 08/16 04:00:03. 0000000299 Disconnect:  SA connHandle: 3  SA connID: 21  IQ connID: 0000000299  User: DBA
  4. I. 08/16 04:00:52. 0000000212 Disconnect:  SAconnHandle: 2  SA connID: 25  IQ connID: 0000000212  User: DBA
  5. I. 08/16 04:05:00. 0000000804 Connect:  SA connHandle: 1000000801  SA connID: 25  IQ connID: 0000000804  User: DBA
  6. I. 08/16 04:05:00. 0000000804 Disconnect:  SA connHandle: 1000000801  SA connID: 25  IQ connID: 0000000804  User: DBA
  7. I. 08/16 04:10:00. 0000000957 Connect:  SA connHandle: 1000000954  SA connID: 23  IQ connID: 0000000957  User: DBA
  8. I. 08/16 04:10:00. 0000000957 Disconnect:  SA connHandle: 1000000954  SA connID: 23  IQ connID: 0000000957  User: DBA
  9. I. 08/16 04:15:00. 0000001109 Connect:  SA connHandle: 1000001106  SA connID: 20  IQ connID: 0000001109  User: DBA
  10. I. 08/16 04:15:00. 0000001109 Disconnect:  SA connHandle: 1000001106  SA connID: 20  IQ connID: 0000001109  User: DBA
  11. I. 08/16 04:20:00. 0000001260 Connect:  SA connHandle: 1000001257  SA connID: 16  IQ connID: 0000001260  User: DBA
  12. I. 08/16 04:20:00. 0000001260 Disconnect:  SA connHandle: 1000001257  SA connID: 16  IQ connID: 0000001260  User: DBA
  13. I. 08/16 04:25:00. 0000001411 Connect:  SA connHandle: 1000001408  SA connID: 11  IQ connID: 0000001411  User: DBA
  14. I. 08/16 04:25:00. 0000001411 Disconnect:  SA connHandle: 1000001408  SA connID: 11  IQ connID: 0000001411  User: DBA
  15. I. 08/16 04:29:16. 0000001541 Connect:  SA connHandle: 4  SA connID: 18  IQ connID: 0000001541  User: DBA
  16. I. 08/16 04:30:00. 0000001564 Connect:  SA connHandle: 1000001560  SA connID: 9  IQ connID: 0000001564  User: DBA
  17. I. 08/16 04:30:00. 0000001564 Disconnect:  SA connHandle: 1000001560  SA connID: 9  IQ connID: 0000001564  User: DBA
  18. I. 08/16 04:30:29.0000001580 Connect:  SA connHandle: 5  SA connID: 27  IQ connID: 0000001580  User: DBA

IQ Shared Temp

$
0
0

IQ Shared Temporary Store Shared dbspace is available from  SAP IQ 15.3 and above. IQ Shared Temporary Store is automatically

created when new database is created or database is upgraded to IQ 15.3 or above. Newly created IQ_SHARED_TEMP does not

include any dbfiles.  Administrator will need to add dbfiles to IQ_SHARED_TEMP  using:

 

                           ‘Alter dbspace  IQ_SHARED_TEMP add file’

 

Shared raw device storage is required for IQ Shared Temporary Store, which should be accessible  and writeable by all Nodes in
the IQ Multiplex. IQ Shared Temporary Store(IQ_SHARED_TEMP) is used to store intermediate results between IQ Multiplex Nodes

during Distributed Query Processing(DQP).

 

The IQ Shared System Temporary (IQ_SHARED_TEMP) dbspace cannot be dropped. The last  dbfile in the IQ_SHARED_TEMP
dbspace can be dropped or made in READONLY in the multiplex single node mode(-iqmpx_sn 1). The IQ_SHARED_TEMP dbspace

cannot be set to READONLY. Temporary user tables cannot be created in IQ_SHARED_TEMP using the ‘IN IQ_SHARED_TEMP’
clause.

  • To add dbfiles to the Shared Temporary Store:

 

                         alter dbspace IQ_SHARED_TEMP add file <logical file name> ‘<physical file path>’

 

  • To drop dbfiles from the Shared Temporary Store:

 

                         alter dbspace IQ_SHARED_TEMP drop file <logical file name>

 

  • To alter Shared Temporary Store dbfile read/write status:

 

      alter dbspace IQ_SHARED_TEMP alter file <logical file name> [READONLY | READWRITE]

 

For complete syntax see:

 

                 http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc00801.1602/doc/html/san1281564739089.html

 

IQ Shared Temporary dbspace dbfiles are added as READONLY file status Once dbfile is added to IQ Shared Temporary dbspace,

all Secondary nodes in the multiplex attempt to access the dbfile and send the file status to the Coordinator. Once all Secondary Nodes

send valid file status to the Coordinator, dbfile can be altered to READWRITE file status and can be used for DQP. Attempt to alter file status to  READWRITE without a VALID file status from all Multiplex Secondary Nodes will result in an SQL error.

 

         
(DBA)> alter dbspace IQ_SHARED_TEMP alter  file iqsharedtmpdsp2 READWRITE

     
      Could not execute statement.

    
      File may not be accessible from one or more multiplex servers. Use   sp_iqmpxfilestatus for further diagnosis.

  
      -- (slib/s_db.cxx 3026)

  
     SQLCODE=-1009192, ODBC 3 State="HY000"

 

   Line 1, column 1   Press ENTER to continue...

 

The sp_iqmpxfilestatus stored procedure can be used to check file status. If executed on Coordinator node, it displays file status for Coordinator and for
every shared dbspace dbfile on every included Secondary Node.  If executed on Secondary Node, it only displays file status for only that Node.

 

  • File Status States:

            - VALID:  file path access and permissions are correct

            - INVALID_PATH: file path name access problem

            - INVALID_PERM – Operating System file permissions are incorrect.

 

In certain scenarios when one of the Secondary Node is down,  DBA can bring IQ SHARED TEMPORARY dbspace dbfile online by bypassing

file status check with the  FORCE READWRITE clause


           alter dbspace IQ_SHARED_TEMP alter file <logical file name> FORCE READWRITE


SAPIQ database options MAX_TEMP_SPACE_PER_CONNECTION and QUERY_TEMP_SPACE_LIMIT limits the sum of local and shared temporary space used by a user connection.

  • If the total of local and shared temporary space exceeds the quota set for user connection, DQP query will fail with “Temporary space quota

        exceeded” error.

  • In case for DQP query, temporary space used is total temporary space used by all nodes participating in the DQP query.
  • The IQ Shared Temporary Store(IQ_SHARED_TEMP) and Local temporary Store(IQ_SYSTEM_TEMP) uses temporary cache configured

        using –iqtc startup switch

 

IQ 16 allows temporary data in IQ_SHARED_TEMP from both temporary and global temporary tables. On Multiplex Systems logical server policy option TEMP_DATA_IN_SHARED_TEMP governs temporary tables creation in IQ_SHARED_TEMP,  default value is ‘off’. Changing this option

requires IQ server to be restarted. ‘with stop server’ clause can be used to automatically shutdown the affected nodes in logical server for

whom ls policy option is changed.

 

  •   If ‘TEMP_DATA_IN_SHARED_TEMP’ login policy server option is set to ‘off’, temporary objects are created in IQ_SYSTEM_TEMP.
  •   If ‘TEMP_DATA_IN_SHARED_TEMP’ login policy server option is set to ‘on’, temporary objects are created in IQ_SHARED_TEMP.


          Example:


                         - alter ls policy lp1 TEMP_DATA_IN_SHARED_TEMP=on with stop server

                         - alter logical server ls1 policy lp1 with stop server

                         - drop logical server ls1 with stop server


sp_iqsharedtempdistrib:


           This system stored  procedure reports the current shared temp space usage distribution. When sp_iqsharedtempdistrib is executed from

           Coordinator, it reports shared temp space distribution for all nodes. When sp_iqsharedtempdistrib is executed from Secondary nodes,

           it reports shared temp space usage for only that node.  Shared temporary space is reserved on each node in the multiplex on demand by

           the  Coordinator. Shared temporary space is reserved for a node in an allocation unit. Nodes can request and hold multiple allocation units

          based on their current demand. These allocation units are leased by the nodes to use more space as needed and return the space to a global

          pool when they don’t need it. Allocation units expire when the shared  temporary space usage decreases, or lease expires or when nodes

          are shutdown.


sp_iqstatus and sp_iqspaceused:


       When these system stored procedures are executed on Coordinator node, they report total space in use on IQ_SHARED_TEMP. When
      executed on Secondary nodes, these system stored procedures do not report space used by shared dbspace ie. IQ_SYSTEM_MAIN,

      IQ_SHARED_TEMP and user dbspaces. sp_iqspaceused takes following arguments:


         sp_iqspaceused(out mainKB            unsigned bigint,

          out mainKBUsed        unsigned bigint,

          out tempKB            unsigned bigint,

           out tempKBUsed        unsigned bigint,

           out shTempTotalKB     unsigned bigint,

           out shTempTotalKBUsed unsigned bigint, 

           out shTempLocalKB     unsigned bigint, 

           out shTempLocalKBUsed unsigned bigint, 

           out rlvLogKB          unsigned bigint,  

           out rlvLogKBUsed      unsigned bigint)

 

User-defined stored procedure myspace can be created that declares all seven sp_iqspaceused output parameters and then call sp_iqspaceused.

For myspace stored procedure code see:

        

            http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc38151.1602/doc/html/san1278453328180.html

 

sp_iqdbspace and sp_iqfile:


   sp_iqdbspace and sp_iqfile system stored procedure report include information for the IQ_SHARED_TEMP.


sp_iqcheckdb

 

     sp_iqcheckdb system stored procedure reports allocation information for IQ_SHARED_TEMP


                     sp_iqcheckdb ‘allocation dbspaceIQ_SHARED_TEMP’


      When executed from Coordinator Node, displays global block usage:


                 -  Blocks Total

                  -  Blocks Used

 

      When executed from any secondary node, sp_iqcheckdb reports blocks reserved for that secondary node:


                - Blocks Reserved for this Server Total

Viewing all 123 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>